ArticlePDF Available

Abstract and Figures

The Youth Program Quality survey, a 24-item survey of youth participant perceptions of program quality, based on program elements identified by the National Research Council (NRC) and Institute of Medicine, was developed and piloted with 614 younger teens (ages 10–13 years) and 486 older teens (ages 14–17 years) who attended 4-H camps and conferences. Evidence is presented for content and construct validity. In addition, the overall instrument demonstrated high reliability, as measured by Cronbach's alpha, generally ranging from .70 to .96, and moderate subscale reliability of .60 or higher on four factors in the younger sample (α=.60 or higher on four factors in the older sample). Results are discussed in terms of goodness of fit to the National Research Council model, the significance of youth voice in assessment of program quality, developmental differences in perceptions of quality, and programming design. Recommendations are offered for practice, research, and policy.
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
Download by: [] Date: 22 September 2015, At: 08:17
Marriage & Family Review
ISSN: 0149-4929 (Print) 1540-9635 (Online) Journal homepage:
Youth Program Quality Survey: Youth Assessment
of Program Quality
Benjamin Silliman & Walter R. Schumm
To cite this article: Benjamin Silliman & Walter R. Schumm (2013) Youth Program Quality
Survey: Youth Assessment of Program Quality, Marriage & Family Review, 49:7, 647-670, DOI:
To link to this article:
Published online: 23 Sep 2013.
Submit your article to this journal
Article views: 111
View related articles
Youth Program Quality Survey:
YouthAssessment of Program Quality
Department of Youth, Family, and Community Sciences,
North Carolina State University, Raleigh, North Carolina, USA
School of Family Studies and Human Services, Kansas State University,
Manhattan, Kansas, USA
The Youth Program Quality survey, a 24-item survey of youth par-
ticipant perceptions of program quality, based on program elements
identified by the National Research Council (NRC) and Institute of
Medicine, was developed and piloted with 614 younger teens (ages
10–13 years) and 486 older teens (ages 14–17 years) who attended
4-H camps and conferences. Evidence is presented for content and
construct validity. In addition, the overall instrument demonstrated
high reliability, as measured by Cronbach’s alpha, generally rang-
ing from .70 to .96, and moderate subscale reliability of .60 or
higher on four factors in the younger sample (
= .60 or higher on
four factors in the older sample). Results are discussed in terms of
goodness of fit to the National Research Council model, the signifi-
cance of youth voice in assessment of program quality, developmental
differences in perceptions of quality, and programming design.
Recommendations are offered for practice, research, and policy.
KEYWORDS adolescence, children, family life education, program
Program quality, particularly contextual features that facilitate support and
challenge, is the best predictor of positive youth development outcomes
Address correspondence to Benjamin Silliman, Box 7606, Department of Youth, Family,
and Community Sciences, North Carolina State University, Raleigh, NC 27695, USA. E-mail:
Marriage & Family Review, 49:647–670, 2013
Copyright © Taylor & Francis Group, LLC
ISSN: 0149-4929 print/1540-9635 online
DOI: 10.1080/01494929.2013.803010
Downloaded by [] at 08:17 22 September 2015
648 B. Silliman and W. R. Schumm
such as preventing risks and building capabilities (Catalano, Berglund, Ryan,
Lonczak, & Hawkins, 2002; Durlak, Weissberg, & Padan, 2010; National
Research Council [NRC] & Institute of Medicine [IOM], 2002; Gambone, Klem,
& Connell, 2002; Mihalic, 2004; Yohalem & Wilson-Ahlstrom, 2010). These
features have been identified consistently in studies of research-based pro-
grams of extended length and likely apply to planned programs of any
length (e.g., one time, short term, and long term). Measures of quality are
often not included in research reports (Yohalem & Wilson-Ahlstrom, 2010)
and are less often assessed among community-based programs, perhaps due
to the limited funding and expertise or lack of sponsor demand for such
results (Eccles & Templeton, 2000; Kahn, Bronte-Tinkew, & Theokas, 2008;
Yohalem & Wilson-Ahlstrom, 2010). Moreover, published reports typically
reflect adult rather than youth participant perceptions of program quality
(Urban, 2008; Yohalem & Wilson-Ahlstrom, 2010). Thus, an accurate, inex-
pensive, youth-friendly measure of program quality might help program
leaders more effectively listen to youth participants and share youth percep-
tions of program quality with key stakeholders, including families with youth
considering or enrolled in such programs. In addition, demonstrating
program quality may augment accountability to and education of program
sponsors regarding “what makes a difference” in promoting program-specific
and broader developmental outcomes for children and adolescents.
This research report describes the development and testing of the Youth
Program Quality (YPQ) survey, a participant rating of experiences in youth
programs. YPQ focuses on the contextual features of youth programs shown
to promote positive youth development by the NRC and the IOM (2002).
These features include safety, support, social norms, social inclusion, skill-
building opportunities, self-efficacy, structure, and synergy with family and
community. The survey was developed to provide a reliable, research-based,
user-friendly tool to help youth leaders incorporate youth voice in assess-
ment of the features in the process of program development, monitoring,
and improvement.
Youth Program Paradigms
A diverse array of youth development programs serve millions of young
people from preschool through early adulthood, nationwide and internation-
ally (NRC & IOM, 2002; Mahoney, Larson, Eccles, & Lord, 2007). These pro-
grams operate on implicit or intentional goals that emphasize risk prevention
(Catalano, et al., 2002; Cicchetti & Rogosch, 2003) and/or positive youth
development (Lerner, etal., 2005; Roth & Brooks-Gunn, 2003; Theokas etal.,
2005). Prevention and developmental models reflect diverse theory and prac-
tice models with many parallel objectives and strategies but share the convic-
tion that “development can be redirected onto a positive course by changing
the pattern of person-to-context relations” (Schwartz, Pantin, Coatsworth, &
Szapocznik, 2007, p. 122).
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 649
Both risk prevention and positive youth development approaches draw
on social ecology or developmental systems theories at the conceptual level
and use social learning, cognitive behavioral, and other theories at the pro-
grammatic level (NRC and IOM, 2002; Schwartz, etal., 2007). Both approaches
recognize that contextual features or processes in the program, program
outcomes, and the larger social environment are reciprocally related. In the
risk prevention model, for instance, a program offering physical and emo-
tional safety and support provides a respite from risks such as bullying and
violence and a context for learning conflict-resolution skills that might serve
as a protective factor in handling future threats. From a positive youth devel-
opment perspective, youth–adult partnerships grounded in adult support,
skill-learning opportunities, and norms of responsibility and service enable
youth to gain connection, character, competence, caring, confidence, and
eventually make contributions to their communities (Lerner etal., 2005).
Youth Program Quality
YPQ is widely regarded as the best predictor of risk prevention and positive
developmental outcomes (Catalano etal., 2002; Durlak etal., 2010; NRC &
IOM, 2002; Larson & Walker, 2009; Lerner etal., 2005; Yohalem & Wilson-
Ahlstrom, 2010). Program participation alone is not sufficient, and programs
low on qualities such as support or skill-building are unlikely to reduce
risks or promote positive outcomes (Larson, 2000; Moore & Hamilton, 2010).
Rather, the interactive climate within programs (Larson, 2000; Durlak etal.,
2010; Larson & Walker, 2009) as well as the ecology of sponsoring organiza-
tions and communities (Ramey & Rose-Krasnor, 2012; Edna McConnell
Clark Foundation, 2012) provide the grounding for long-term program
YPQ is described in at least three ways. A comprehensive view of YPQ
(Edna McConnell Clark Foundation, 2012; Reisner, White, Russell, & Birmingham,
2004) includes the broader organizational capacity for management; engage-
ment of youth, families, and community partners; staff development; evi-
dence-based activities; evaluation; and sustainability, while offering a sup-
portive and challenging climate for youth. While acknowledging the
importance of a broader context, the organizational view of YPQ exempli-
fied in the High/Scope model (2012) focuses on organizational or site poli-
cies, expectations, and access, together with a point-of-service elements of
safety, support, interaction, and engagement indicators. Similarly, Huang and
Dietel (2011) identified key elements of good afterschool programs as (1)
goals, reflected in structured experiences; (2) leadership, characterized by
horizontal and vertical communication and collaboration; (3) capable and
caring staff; and (4) opportunities for skill-building and self-efficacy. Durlak
and DuPre (2008) linked quality of management and environment to other
dimensions of program fidelity such as dosage, participant responsiveness,
and staffing quality.
Downloaded by [] at 08:17 22 September 2015
650 B. Silliman and W. R. Schumm
A program-oriented view of YPQ, typical of most reports of program
quality, focuses on the environment and interactions “on the ground” within a
program (NRC & IOM, 2002; Gambone et al., 2002; Yohalem & Wilson-
Ahlstrom, 2010). Collating findings from two decades of prevention and enrich-
ment program research, the NRC and IOM (2002) identified eight contextual
features within programs that are most likely to promote positive developmen-
tal outcomes: (1) safety, physical, and emotional; (2) support from adults and
peers; (3) social norms, including respect and service; (4) social inclusion via
acceptance and belonging; (5) skill-building opportunities; (6) self-efficacy
and contribution to others; (7) structure appropriate to audience and setting;
and (8) synergy between youth programs and family, neighborhood, and com-
munity life. Subsequent research supported and expanded this description.
For instance, Roth and Brooks-Gunn (2003) identified quality components as
active citizenship, opportunities for skill-building and leadership, promotion of
wellness and future-time perspectives, fun activities, sustained youth–adult
relationships, and a sensitive and inclusive environment.
Lerner (2004) identified “The Big Three” features of effective youth-
serving programs as (1) positive and sustained relationships between youth
and adults, (2) activities that build important life skills, and (3) opportunities
to use life skills as participants and leaders. Similar findings are reflected in
research reviews on informal science (Friedman, 2008), residential camps
(Henderson, Bialeschki, Thurber, Schueler, & Marsh, 2007; Scanlon, 2006),
afterschool programs (Granger, Durlak, Yohalem, & Reisner, 2007), and civic
engagement (Michelson, Zaff, & Hair, 2002). Yohalem and Wilson-Ahlstrom
(2010) noted that recent research across teen and preteen programs under-
lines the roles of positive relationships, supportive climate, and consistent
engagement of youth as contributors to social and academic outcomes.
Sabatelli, Anderson, Kosutic, Sanderson, and Rubenfeld (2009) identified
four key concepts, namely, safety, support, challenging activities, and mean-
ingful relationships, from literature on quality. They created and critically
reviewed a 65-item questionnaire, and then conducted a pilot test with after-
school programs (serving 6th–12th graders) as the units of analysis, applying
exploratory factor analysis, which yielded three factors: supportive involve-
ment, challenge and involvement, and emotional well-being and safety. A
subsequent field test applying confirmatory factor analysis with a similar
audience identified several three- and two-factor models, depending on the
analytical approach. Process use was also applied to program improvement.
More recently, Tiffany, Exner-Cortens, and Eckenrode (2012) tested a
20-item measure with 12- to 18-year-old youth in urban afterschool programs
that yielded four key components of youth program participation: personal
development, voice and influence, safety and support, and community
engagement. Again, components identified resemble those featured in the
NRC-IOM report (2002), with emphasis on program input and community
involvement for older youth.
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 651
Youth Voice in Program Evaluation
Program quality is best documented using multiple measures from multiple
sources over multiple points through the course of a program (Yohalem &
Wilson-Ahlstrom, 2010). Because youth voice is a critical element in the phi-
losophy and practice of positive youth development (Camino & Zeldin, 2002;
Hamilton, Hamilton, & Pittman, 2004), youth perspectives represent a critical
starting point in quality assessment. Moreover, participant readiness and
responsiveness is a critical indicator of program fidelity (Carroll etal., 2007).
Positive youth development and YPQ are most often defined and measured by
adults, whose views of programs vary somewhat from youth. For instance,
Urban (2008) found that although youth generally agreed with adult practitio-
ners, policymakers, and researchers on the most critical traits of positive youth
development programs (e.g., youth-friendly environments, opportunities for
leadership and life skills practice, balance of individuality and connectedness),
youth and researchers held different priorities on youth input and evaluation.
From a practical perspective, youth are the most authentic evaluators of
their experience. Youth leaders who know how to evoke and discern youth
feedback can be more effective in planning, monitoring, and improving pro-
gramming, thus aiding the achievement targeted outcomes. Youth and adults
offer complementary voices (Larson, Walker, & Pearce, 2004) that result in
more accurate and complete view of a program’s quality and impact. In the
absence of multiple measures, a brief but thorough participant survey pro-
vides an efficient, if not sufficient, measure of whether and how a program
is working. Although positive voices for program quality may not be inferred
as evidence for program outcomes, negative views might serve as formative
evidence for program fidelity and corroborative evidence that summative
outcomes rest on sound practices (Yohalem & Wilson-Ahlstrom, 2010).
This research reports on the development and testing of a measure of
YPQ, as perceived by preteen and teen participants in several 4-H camps
and conferences. Prior studies with smaller samples indicated high program
quality and instrument reliability as well as positive knowledge growth and
behavioral outcomes (Silliman, 2008; Silliman & Shutt, 2010). This research
incorporates a broader sample and then applies exploratory factor analysis
to determine salient elements of program quality for younger and older
The YPQ survey was implemented over a 3-year period as part of routine
program evaluation for weeklong and 3-day citizenship and service con-
ferences and in weeklong resident camps sponsored by a 4-H program in
Downloaded by [] at 08:17 22 September 2015
652 B. Silliman and W. R. Schumm
the southeastern United States. 4-H is a nationwide youth development
organization serving 5- to 19-year-olds through programs in citizenship,
science, and healthy living, among other issues (4-H National Headquarters,
2012). Participation in the research study was voluntary, but 95% of par-
ents and youth completed a human subjects consent form previously
approved by the first author’s institutional review board. No parents or
youth refused participation, but 5% omitted the consent form in event
registration. Only data from consenting parents and youth were used in
this study. To reduce heterogeneity in age and development within groups,
participants younger than 10 years or older than 17 years were not included
in the analyses presented here.
In the younger sample (ages 14–17) of 614 participants, there were
more boys (52.0%) than girls (48.0%). Average age was 11.67 years (standard
deviation [SD] = 1.04), with more participants aged 12 years (30.5%) than
other ages (10 years, 16.3%; 11 years, 26.9%; 13 years, 26.4%). Average grade
level was 6.86 (SD = 0.86), with more participants in grade 7 (37.5%) than
other grades (grade 5, 4.2%; grade 6, 31.9%; grade 8, 26.4%). There were 542
missing cases on the “grade” item. Most participants had attended a 4-H
camp (90.1%) rather than a 4-H conference (9.9%). Most data were missing
for ethnicity (n = 556), but most of those who did respond were White
(84.5%) or Black (10.3%), with a few other responses (Asian, 3.4%; other,
1.7%). Data on ethnicity were only collected in 2008 at two sites; data on
grade level were collected only in 2007 and 2010 at three sites.
In the older sample (ages 14–17) of 486 participants, there were more
girls (66.5%) than boys (33.5%). Average age was 15.19 years (SD = 1.07),
with more participants aged 14 years (35.0%) than other ages (15 years,
26.5%; 16 years, 23.5%; 17 years, 15.0%). Average grade level was 10.27
(SD = 1.17), with more participants in grade 9 (28.8%) than other grades
(grade 7, 1.7%; grade 10, 27.1%; grade 11, 23.7%; grade 12, 18.6%). There
were 78 missing cases on the “grade” item. Most participants had attended a
4-H conference (82.1%) rather than a 4-H camp (17.9%). Most data were
missing for ethnicity (n = 409), but most of those who did respond were
White (71.4%) or Black (18.2%), with a few other responses (White, Hispanic,
1.3%; Hispanic, 6.5%; Asian, 1.3%; other, 1.3%).
Event registration forms provided demographic information used in group
comparisons. The YPQ was designed to evaluate participant perceptions of
experiences in short- and long-term youth programs. Three items were
developed for each of eight contextual features that promote positive youth
development (NRC & IOM, 2002), resulting in a 24-item survey (see Table 1
for items by trait). Survey items were reviewed for reading comprehension
by panels of youth and adult leaders and piloted with youth groups before
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 653
TABLE 1 YPQ Survey with NRC-IOM (2002) Traits Identified in Brackets
disagree Disagree Agree
1. I felt safe from being hurt or
2. I was embarrassed or put down……… A B C D
3. Activities promoted healthy habits……… A B C D
4. Adults listened to what I had to say……… A B C D
5. I felt comfortable going to adults for
6. Other kids cared about me……… A B C D
[Social Norms]
7. Conflicts between people were a
8. I learned to work with others as a
9. Serving others and volunteering was
[Social Inclusion]
10. I felt like I didn’t belong……… A B C D
11. All kinds of kids were welcomed……… A B C D
12. I learned to accept differences in
[Skill-building Opportunities]
13. Activities taught me to develop a plan to
reach my goals………
14. I was challenged to think and build
15. There were opportunities to learn new
16. I felt that I could make a difference……… A B C D
17. I was encouraged to take
18. Perfect performance was more important
than learning from mistakes………
19. Rules and expectations were clear……… A B C D
20. Discipline was not too strict, not too
21. Activities were just right for my age……… A B C D
22. I gained a broader view of 4-H……… A B C D
23. I gained a broader view of the world
beyond my community………
24. Activities were related to issues in my
club, my family, my community………
drafting a final edition. Youth feedback resulted in changing Likert-type
response options from “never–seldom–often–always” to a “strongly disagree–
strongly agree” continuum. Straightforward statements were preferred over
Downloaded by [] at 08:17 22 September 2015
654 B. Silliman and W. R. Schumm
negatively worded items, which youth and their leaders deemed too confus-
ing for inexperienced respondents. Thus, only four items were negatively
worded or reverse-coded.
Content validity was established on elements of program quality
described by the NRC and IOM report (2002), widely regarded as best prac-
tices in the field (Yohalem & Wilson-Ahlstrom, 2010). Although these internal
assets may be described with different labels, similar elements can be found
in youth programming at camps (Henderson etal., 2007), afterschool pro-
grams (Granger et al., 2007), and informal science programs (Friedman,
2008). A 1999 national study of program quality by the 4-H youth develop-
ment organization identified similar elements, although they were labeled
differently (Peterson, 2001).
Construct validity, the proposition that a measure accurately reflects
what it is intended to measure, was initially established through panel review
and pilot study in which youth and adult volunteers and professionals sup-
ported its face validity or perceived consistency with the construct of pro-
gram quality. Exploratory factor analysis was then conducted to evaluate
observed patterns in the data relative to NRC-IOM model constructs. Separate
analyses were conducted for younger and older adolescent samples consis-
tent with developmental differences in social cognition (Lerner, 2009). Results
of these analyses are reported below.
Conference attendees completed paper forms, whereas camp participants
completed either paper forms or used clickers to respond to PowerPoint
slides read by a trained proctor. Monitoring and debriefing discussions with
program staff indicated that participants had few difficulties completing the
survey in either mode. If lower-literacy participants experienced difficulty
with the paper survey, the staff was instructed to read items aloud without
elaboration. About 10 participants required such assistance.
Given the developmental differences between teens and preteens (Lerner,
2009), data from program participants 10–13 years of age and 14–17 years
of age were analyzed separately. The 24 items in the YPQ scale were sub-
jected to exploratory factor analysis, using maximum likelihood factoring,
extracting four to six factors in search of the best apparent solutions.
Factoring was conducted both with and without substitution for missing
data; because the solutions were similar, results for the larger sample with
mean substitution for missing data are presented here. A ratio of 20:1 for
subjects-to-items was exceeded, meeting the most strict subject-to-item
requirements recommended (Hair, Black, Babin, Anderson, & Tatham,
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 655
TABLE 2 Varimax-Rotated Maximum Likelihood Factor Loadings for Items from the YPQ
Survey: Younger Youth Sample (Ages 10–13) (n = 610)
Rotated factor loadings
12 3 4 5h2
5. I felt comfortable going to adults
for advice.
.71 .01 .09 .13 –.05 .54
4. Adults listened to what I had to
.71 .13 .09 .13 .00 .55
8. I learned to work with others as a
.54 .31 .22 –.03 –.15 .46
1. I felt safe from being hurt or
.52 .17 .03 .15 –.03 .32
6. Other kids cared about me. .51 .17 .16 .04 –.19 .36
11. All kinds of kids were welcomed. .44 .21 .17 .24 –.12 .34
9. Serving others and volunteering
was important.
.43 .28 .18 .15 –.16 .34
12. I learned to accept differences in
.41 .34 .17 .19 –.21 .39
3. Activities promoted healthy habits. .38 .34 .22 .07 .03 .31
14. I was challenged to think and
build new skills.
.15 .56 .20 .18 .07 .41
13. Activities taught me to develop a
plan to reach my goals.
.26 .55 .29 .07 –.05 .47
15. There were opportunities to learn
new subjects.
.27 .47 .19 .25 –.02 .39
17. I was encouraged to take
.21 .36 .30 .13 –.13 .30
16. I felt that I could make a
.21 .34 .33 .15 –.05 .30
2006, p. 112; Osborne & Costello, 2004). Goldberg and Velicer (2006) indi-
cate that “samples in the 500–1,000 range are preferred” (p. 214) for factor
Cronbach’s (1951) alpha, a “widely used estimate of reliability” (Zeller
& Carmines, 1980, p. 59), or internal consistency was used to assess survey
data for each contributing event and scales derived from the factor analysis.
SPSS (Norusis, 2006) was used for computing both the factor analyses and
the alphas.
Factor Analysis for Younger Youth (Preteen) Sample
Exploratory factor analyses were conducted on data from younger (ages 10–13)
and older (ages 14–17) youth. For younger youth, five factors were produced with
eigenvalues > 1.0 with eigenvalues between 1.05 and 6.66 (Table 2). The Kaiser-
Meyer-Olkin (KMO) measure of sampling adequacy was 0.91 with a Bartlett Test
Downloaded by [] at 08:17 22 September 2015
656 B. Silliman and W. R. Schumm
of Sphericity of 3,165.1, df = 276, p < .001. Extracted communalities ranged between
.11 and .60. Attempts to extract fewer factors yielded more residuals (>.05) from
the reproduced correlation matrix, 6% for four factors. Attempts to extract more
than five factors yielded minor factors with loadings on only one item, with 1%
residuals. Use of listwise deletion for missing data yielded similar factors with 6%
residuals (>.05). All items loaded at least 0.32 on at least one factor.
The first factor for the younger youth consisted of nine items related to
NRC-IOM Safety and Health (items 1, 3), Adult and Peer Support (items 4, 5,
6), Social Norms (items 8, 9), and Social Inclusion (items 11, 12) and was
labeled “Positive Emotional Climate” YPQ item numbers are listed after each
factor label for results of Younger Youth and Older Youth factor analyses.
The second factor, “Empowered Skill-building,” included three items from
NRC-IOM Skill-building (13, 14, 15) and two from NRC-IOM Self-efficacy (16,
17). The third factor, “Expanding Horizons,” included items from NRC-IOM
“Synergy” (22, 23, 24). The fourth factor included YPQ items 19, 20, and 21
related to “Structure” and was labeled accordingly. The fifth factor included
four items (2, 7, 10, and 18) from different NRC-I OM categories and was
labeled “Negative Experiences.”
23. I gained a broader view of the
world beyond my community
.13 .23 .71 .17 –.08 .60
22. I gained a broader view of 4-H. .14 .23 .62 .20 –.04 .50
24. Activities were related to issues in
my club, my family, and my
.14 .21 .41 .16 .03 .26
20. Discipline was not too strict, not
too loose.
.16 .15 .17 .57 –.08 .40
21. Activities were just right for my
.19 .14 .33 .49 –.09 .41
19. Rules and expectations were clear. .21 .28 .28 .42 –.06 .38
10. I felt like I didn’t belong. –.18 –.01 –.07 .01 .60 .39
2. I was embarrassed or put down. –.10 .00 –.02 –.03 .52 .29
7. Conflicts between people were a
–.07 –.00 –.10 –.05 .46 .23
18. Perfect performance was more
important than learning from
.05 –.03 .06 –.04 .32 .11
Eigenvalues 6.66 1.84 1.55 1.08 1.05
KMO sampling adequacy 0.91
Bartlett Test of Sphericity 3,169.1, p < .001 (df = 276)
Reproduced correlation matrix
residuals > .05
Note. Primary factor loadings are in bold type. Eigenvalues are associated with unrotated factors only.
Communalities (h2) are extracted communalities.
TABLE 2 Continued
Rotated factor loadings
12 3 4 5h2
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 657
Factor Analysis for Older Youth (Teen) Sample
An identical factor analysis was conducted for the older sample. Use of an
eigenvalue = 1.0 criterion with maximum likelihood factor analysis yielded
five factors, with eigenvalues between 1.03 and 7.72 (Table 3). The KMO
measure of sampling adequacy was 0.93 with a Bartlett Test of Sphericity of
3,316.3, df = 276, p < .001. Extracted communalities ranged between .10 and
.66. Attempts to extract fewer factors yielded more residuals (>.05) from the
reproduced correlation matrix, 13% for four factors. Attempts to extract more
than five factors yielded minor factors with loadings on only one item, with
3% residuals. Use of listwise deletion for missing data yielded similar factors
with 8% residuals (>.05). All items loaded at least 0.30 on at least one factor,
but several apparent factors included items with loadings < .40. Item 20 was
the only item that did not load at least 0.33 on at least one factor.
The first factor loaded heavily on five items related to “Empowered
Skill-building” and included items from NRC-IOM traits of Skill learning oppor-
tunity (13, 14, 15), responsibility Social Norms (17), and developmentally appro-
priate Structure (17). Removing item 21 (Structure) from the scale only reduced
Cronbach alpha from .79 to .77. Only the items loading > .40 were retained
for the subscale derived from the four items with the heaviest loadings on the
first factor. The second factor loaded primarily on seven items and was labeled
“Positive Values” because the items seemed to focus on YPQ items for Social
Norms (8, 9), Social Inclusion (11, 12), Structure (19), and Safety (1, 3) related
to an inclusive environment in which service and teamwork were valued. The
third factor loaded on four items, including YPQ Self-Efficacy (16) and Synergy
(22, 23, 24) items, and was labeled “Expanding Horizons” because the items
referred to broadening views of 4-H and the world in context of family and
community. Item 16 related to Self-efficacy, also loaded to some extent on the
first and fourth factors but seemed to best support a higher Cronbach’s alpha
for this subscale. The fourth factor loaded on items 4 and 5 and was labeled
“Adult Support,” consistent with the YPQ item. The fifth factor related to
“Negative Experiences” loaded on five items consistent with NRC-IOM
descriptors, but the subscale generated from items 2, 6 (recoded), 7, 10, and
18 had an unacceptably low Cronbach’s alpha (.56).
The factor structures were somewhat similar for the two age groups.
Items 22, 23, and 24 formed a distinct factor; indeed, the third factor was
labeled “Expanding Horizons” in both groups, with item 16 (“Making a
Difference”) loading on the same factor for the older sample. Cronbach’s alpha
for this subscale was a bit higher (.78) for the older group than for the younger
group (.69). The fifth factor for both groups was labeled “Negative Experiences”
and included items 2, 7, 10, and 18 for both groups, although item 6 was
included for the older group. However, for both subscales, Cronbach’s alphas
were low (.56). The relatively low average scores for both Negative Experiences
subscales may suggest that few participants experienced rejection, conflict, or
Downloaded by [] at 08:17 22 September 2015
658 B. Silliman and W. R. Schumm
TABLE 3 Varimax-Rotated Maximum Likelihood Factor Loadings for Items from the YPQ
Survey: Older Youth Sample (Ages 14–17) (n = 486)
Rotated factor loadings
12 3 4 5 h2
13. Activities taught me to develop a plan
to reach my goals.
.64 .15 .30 .16 .11 .55
14. I was challenged to think and build
new skills.
.58 .16 .22 .27 .05 .48
17. I was encouraged to take
.50 .14 .30 .18 .27 .46
15. There were opportunities to learn
new subjects.
.47 .33 .19 .18 .09 .41
21. Activities were just right for my age. .37 .32 .11 .23 .16 .33
12. I learned to accept differences in
.08 .61 .28 .20 .28 .58
11. All kinds of kids were welcomed. .21 .57 .17 .16 .21 .47
9. Serving others and volunteering was
.26 .45 .29 .16 .25 .44
19. Rules and expectations were clear. .26 .36 .09 .11 .19 .25
8. I learned to work with others as a
.21 .35 .32 .28 .18 .39
1. I felt safe from being hurt or injured. .18 .35 .01 .24 .34 .33
3. Activities promoted healthy habits. .17 .33 .18 .33 .16 .30
23. I gained a broader view of the world
beyond my community.
.30 .15 .70 .14 .17 .66
22. I gained a broader view of 4-H. .23 .23 .59 .15 .06 .47
24. Activities were related to issues
in my club, my family, and my
.41 .16 .50 .06 .10 .46
16. I felt that I could make a
.38 .23 .38 .30 .15 .46
4. Adults listened to what I had
to say.
.22 .17 .09 .66 .10 .52
5. I felt comfortable going to adults for
.19 .18 .14 .65 .26 .58
20. Discipline was not too strict, not too
.24 .20 .22 .31 .05 .25
10. I felt like I didn’t belong. .16 .14 .07 .01 .59 .40
7. Conflicts between people were a
.08 .11 .15 .18 .46 .28
6. Other kids cared about me. .14 .23 .16 .29 .36 .31
18. Perfect performance was more
important than learning from
.06 .08 .15 .10 .35 .17
2. I was embarrassed or put down. .07 .08 .04 .02 .30 .10
Eigenvalues 7.72 1.58 1.20 1.07 1.03
KMO sampling adequacy 0.93
Bartlett Test of Sphericity 3,316.3, p < .001 (df = 276)
Reproduced correlation matrix
residuals >.05
Note. Primary factor loadings are in bold type. Eigenvalues are associated with unrotated factors only.
Communalities (h2) are extracted communalities.
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 659
unfair demands, making the items less relevant or meaningful for most partici-
pants. The second factor for the younger participants and first factor for the
older youth (labeled “Empowered Skill-building”) included the same four
items, 13, 14, 15, and 17, whereas item 16 was also included for the younger
sample. For both groups, Cronbach’s alphas were above .70.
For the younger group, items pertaining to both adult (4, 5) and group/
peer acceptance (1, 3, 8, 9, 11, 12) loaded on the first factor (“Positive
Emotional Climate”) together, whereas for the older group, those items formed
the second (1, 3, 8, 9, 11, 12, 19; “Positive Values”) and fourth (4, 5; “Adult
Support”) factors. For the younger sample, a fourth factor (items 19, 20, and
21; “Structure”) did not correspond to factors found for the older sample. For
the older sample, item 20 did not load strongly on any of the five factors.
Items 19 and 21 loaded, for the older sample, on the second and first factors,
It appears that for the younger youth, the first factor of the YPQ survey
focused on their acceptance by adults and youth participants, whereas
those issues formed two distinct factors, acceptance by adults and peers,
separately, for the older youth. In terms of personal development
(“Empowered Skill-building”), gaining a broader view of the world
(“Expanding Horizons”), and perceived problems (“Negative Experiences”),
the younger and older participants seemed to report similar cognitive per-
ceptions of the scale items and their interrelatedness. Although most items
loaded primarily on one factor, a few showed relatively equal weights on
two factors. For the younger group, item 3 related to promotion of healthy
habits held about the same weight on “Positive Emotional Climate” (.38) and
“Empowered Skill-building” (.34). Loadings may reflect the role of health as
a context and target outcome or suggest that the item was too ambiguous
to discriminate well between those factors. Likewise, item 16, “I felt I could
make a difference,” had similar (although low) loadings for “Empowered
Skill-building” (.34) and “Expanded Horizons” (.33), a parallel with older
youth on the former, but connected to gaining a broader view of youth club
For older youth, item 21 on age appropriateness loaded about equally
for “Empowering Skill-building” (.37) and “Positive Values” (.32), reflecting
the importance of developmental as well as social “fit” for achievement and
relational dimensions of youth programs. Item 24, reflecting the connec-
tion between 4-H activities and club, family, and community life, had high
loadings on both “Adult Support” (.50) and “Empowered Skill-building”
(.41), again reflecting the importance of context for affective and instru-
mental growth. A similar pattern is evident relative to self-efficacy on item
16, “I felt I could make a difference,” which might be seen as an element
on “Empowered Skill-building” (.38) and as a consequence of “Adult
Support” (.38).
Downloaded by [] at 08:17 22 September 2015
660 B. Silliman and W. R. Schumm
Finally, some of the factor analysis results for the younger children may
reflect a methods effect as the younger children’s factors tended to follow the
order of the questions: factor 1, items 3, 4, 5, 6, 8, 9, 11, and 12; factor 2,
items 13, 14, 15, 16, and 17; factor 3, items 22, 23, and 24; factor 4, items 19,
20, and 21; and factor 5, items 2, 7, 10, and 18. Thus, the structure of the
questionnaire may have influenced the responses of the younger 4-H mem-
bers more than it did for the older 4-H members. Table 6 summarizes those
items included in each factor for younger and older groups.
Internal consistency as measured by Cronbach’s (1951) alpha for the entire
instrument (24 items) ranged between .70 and .88 for younger participants
and between .77 and .96 for older participants across each 4-H conference
or camp. Combining data from all camps and conferences, alphas were .85
and .82 for the entire instrument (24 items) for the younger and older sam-
ples, respectively. For the two samples, the alpha statistic, along with sub-
scale means and SDs, are presented in Tables 4 and 5.
Overall survey reliability (Cronbach, 1951) was calculated for each con-
ference or camp, with alpha coefficients ranging from .70 to .88 for younger
participants. Reliability of the overall instrument for teens (ages 14–17 years)
at 4-H camps/conferences ranged from
= .77 to .96. For the two samples,
reliability estimates (Cronbach, 1951) along with subscale means and stan-
dard deviations are presented in Tables 3 and 5.
TABLE 4 Characteristics for Subscales of the YPQ Survey: Younger Youth Sample
Subscale No. items nMean SD
Positive Emotional Climate 9 563 29.13 5.40 0.84
Empowered Skill-building 5 614 15.99 3.13 0.74
Expanding Horizons 3 502 9.31 2.23 0.69
Structure 4 508 9.82 2.08 0.64
Negative Experiences 4 501 7.49 2.58 0.56
TABLE 5 Characteristics for Subscales of the YPQ Survey: Older Youth Sample
Subscale No. items nMean SD
Empowered Skill-building 4 447 13.40 2.40 0.77
Positive Values 7 436 24.42 3.38 0.79
Expanding Horizons 4 443 13.39 2.45 0.78
Adult Support 2 474 6.49 1.46 0.71
Negative Experiences 5 440 8.54 2.59 0.56
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 661
TABLE 6 Specific Items and NRC-IOM Traits for Subscales of the YPQ Survey
Subscale Item Numbers NRC-IOM Trait
younger youth sample
Positive Emotional Climate 1, 3 Safety
4, 5, 6 Support
8, 9 Social Norms
11, 12 Social Inclusion
Empowered Skill-building 13, 14, 15 Skill-building Opportunities
16, 17 Self-efficacy
Expanding Horizons 22, 23, 24 Synergy
Structure 19, 20, 21 Structure
Negative Experiences 2, 7, 10, 18 Negatively worded, recoded
older youth sample
Empowered Skill-building 13, 14, 15 Skill-building Opportunities
17 Social Norms
21 Structure
Positive Values 8, 9 Social Norms
11 Social Inclusion
19 Structure
1, 3 Safety
Expanding Horizons 16 Self-efficacy
22, 23, 24 Synergy
Adult Support 4, 5 Support
Negative Experiences 2, 6, 7, 10, 18 Negatively worded, recoded
General Comments
The 24-item YPQ survey was developed to provide a rapid and reliable indi-
cator of youth program quality keyed to participants’ perceptions of their
experience in the program. Internal validity was established for youth aged
10–17, through alignment with research-based best practices, review by youth
and adult leaders, and piloting with teens and preteens. The instrument was
used as part of routine postevent evaluation for several conferences and
camps sponsored by a 4-H program, with data from consenting youth and
parents entered into research analyses. Youth ages 10–17 completed survey
items without difficulty, and program staff found the results useful for pro-
gram accountability and improvement.
Exploratory factor analyses identified five factors with similar themes
for samples of younger youth (ages 10–13) and older youth (ages 14–17).
Scale reliability coefficients (.64 to .84 for younger;
= .71 to .79 for older)
were moderate-to-high (Cronbach, 1951) on four of five factor scales for
both groups. Both groups yielded one factor related to “Negative Experiences”
with unacceptably low reliability (
= .56). Not all youth program experi-
ences are positive (Larson, Hanson, & Moneta, 2006), but negatives may be
rare enough in this sample to decrease reliability. Alternatively, items might
not have been clear to all respondents. An abbreviated version of the YPC
Downloaded by [] at 08:17 22 September 2015
662 B. Silliman and W. R. Schumm
(nine items) used in the 4-H Science, Engineering, and Technology Youth
Engagement, Attitudes, and Knowledge Survey (LeFleur, Milke, & Sanzone,
2010) yielded an
= .83 reliability. Prior studies with subsets of preteens
(Silliman & Shutt, 2010) and teens (Silliman, 2008) found the overall survey
highly reliable (e.g.,
= .88 and .87, respectively). Singletary, Smith, and
Evans (2006) reported alphas between .76 and .87 for similar scales com-
pleted by adults. However, data in that study were not factor-analyzed.
Comments on Specific Groups
Exploratory factor analyses were conducted separately for younger and older
youth. Five factors emerged for the younger preteen group that differed from
the teen sample in the items and order of three factors. Factor 1, “Positive
Emotional Climate,” includes Social Norms (2), Inclusion (2), and Safety and
Health (2) items on the teen “Positive Values” factor along with both adult
and peer Support items (3). Both the priority and broader content of this
factor is consistent with the salience of identity and social adjustment issues
(e.g., being accepted and accepting others, cooperation in teams) for pre-
and early adolescents relative to older adolescents (Lerner, 2009). Although
belonging, safety, and teamwork are important for all youth, mature teens
tend to have a more secure identity, interactive skills, and life experience that
enables them to adjust to or even remove “speed bumps” that arise in social
groups (Lerner, 2009). This complex of elements in a single factor perhaps
illustrates younger youths’ tendency to view the social environment as a
whole, not distinguishing separate qualities such as belonging, service, or
support. Factor 2, “Empowered Skill-building” includes all but one item load-
ing on the similar factor for older youth. Likewise, Factor 3, “Expanding
Horizons,” mirrors the same factor of the older group. The Self-efficacy
“make a difference” item had similar loadings on Factors 2 and 3 that, like its
relation to Adult Support for the older sample, probably represent an infer-
ence about the effects of learning and broadening experiences. Perhaps this
breadth of application diminishes the distinctiveness of Self-efficacy as a
separate factor. Factor 4, “Structure,” coincides with the same NRC element,
and its emergence as a distinct element for younger youth may coincide with
their developmental needs for clearer rules, takes, and organization. Factor
5, “Negative Experiences,” may be an artifact of wording and coding or
reflection of less-than-ideal experiences that earlier analyses showed to be
atypical of the events as a whole (Silliman, 2008; Silliman & Shutt, 2010).
Analyses with older youth also identified five factors that were somewhat,
but not entirely, similar to the traits on which items were built. Factor 1,
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 663
labeled “Empowered Skill-building,” focuses principally on Skill Opportunities
combined with a Social Norm item on responsibility and a Structure item on
age-appropriate activities that are consistent with program qualities that
foster achievement (Gambone etal., 2002; Larson etal., 2004). Skill content
such as “leadership” or “life skills” was not specified as with Urban (2008)
and others. Factor 2, “Positive Values,” includes two items each from Social
Norms, Inclusion, and Safety and one Structure item on clear rules. These
patterns show that teens distinguish a skill-related set from its social context,
which teens view as a complex of prosocial values related to personal growth
(e.g., learning to accept others), group interaction (e.g., safety, teamwork),
and interaction with the community (e.g., serving others). Thus, although
program qualities may be described distinctly—by experts or teens them-
selves—they are experienced holistically. Moreover, values of acceptance
and cooperation are common to both Social Norms and Inclusion and related
behaviors can foster a sense of Safety and consistent Structure. Perhaps the
“mix” of qualities may also shift from one program, setting, or audience to
another, as illustrated by Larson etal. (2006).
The third factor, “Expanding Horizons,” includes only Synergy items for
both groups, making it the most consistent with the NRC-IOM model. The
“broadening” and “in context” elements are reminiscent of autonomy and
connectedness themes evident in youth development and programming
research (Larson, 2000; Lerner, 2009; Urban, 2010). This connection might
also be an artifact of the conference focus on gaining vision and skills for
broader service. Factor 4, “Adult Support,” includes two items on adult roles
together with a Self-efficacy item on “confidence to make a difference,” per-
haps anticipating the consequences of support. However, the “difference”
item had the same loading for Factor 1 and seems to fit better as a correlate
of Skill-building. Encouragement and expectations elements of support
emerge in Factors 1 and 2, yet older youth identify less directive adult roles
of availability and trustworthiness as distinct qualities of their program expe-
rience. This pattern is consistent with previous research on agency (Larson,
2000) and appropriate roles of youth workers (Urban, 2008). As noted above,
“Negative Experiences” represents a “catch-all” of bad experiences and
reverse-worded or coded items.
YPQ priority factors for young adolescents (10–14 years), Supportive
Environment and Empowered Skill-building, are similar to factors identified
by the Youth Development Assessment Device (Sabatelli et al., 2009) in
which two thirds of respondents in both study samples were sixth to eighth
graders. A third factor, Emotional Well-being, included safety and belonging
items found in the YPQ Positive Emotional Climate. The greater number of
factors identified in the YPQ likely reflects the greater diversity of topics (8)
in the YPQ relative to the Youth Development Assessment Device (4). Factor
precedence for older youth more closely resembles participation factors
identified with older youth (12–18 years) by Tiffany etal. (2012) as their
Downloaded by [] at 08:17 22 September 2015
664 B. Silliman and W. R. Schumm
Personal Development factor resembles Empowered Skill-building, Safety
and Support parallels the YPQ Positive Values and Adult Support factors, and
Community Engagement describes the activities that led youth in the current
study to describe programs as Expanding Horizons on self and community.
The youth Voice and Influence factor identified by Tiffany etal. (2012) con-
tains items similar to those in Empowered Skill-building and Positive Values,
but specific items about influence were not asked in the YPQ.
For both younger and older youth, emergent factors broadly resemble scales
derived from the NRC-IOM model, clustering around skill opportunities,
social norms and values, and broadening experiences, with a weaker factor
related to negative experiences. Norm and skill factors emerged in a different
order for each group, with adult and peer support included with normative
behaviors in the view of younger youth. Factors unique to age groups include
Structure among younger youth and Adult Support among older youth.
These patterns are likely reflective of each group’s developmental needs and
capacities. Younger youth may be more dependent on a more explicit
Structure to regulate social and skill-building behavior, whereas older youth
may be more self-regulated, attuned to social norms, and/or involved in
leadership. In that context, Adult Support may be more salient for personal
growth and group dynamics.
Alternatively, age differences may partially reflect emphases in camps,
with a majority younger audience, and conferences, primarily attended by
older youth. Safety, Social Norms, and Self-efficacy did not emerge as sepa-
rate factors, perhaps because they represent broad themes with diverse appli-
cations (e.g., lack of threat or embarrassment, health habits; teamwork, con-
flict resolution, service; responsibility, doing your best, making a difference).
Participants may see these qualities as diffused throughout a program rather
than distinct, viewed as a whole rather as separate qualities. Qualities may
also overlap and complement each other. Qualities such as Structure, Safety,
Social Inclusion, and Social Norms may be most evident when not present
(e.g., disorganization, bullying) and “taken for granted” in smooth-running
programs. The quality of programs surveyed, evident in a high percentage of
positive ratings, moderate-to-high scale reliability, and the relative weakness
of the Negative Experiences factor may diminish the likelihood of detecting
such program qualities. Even when factors have similar items, however,
abbreviated statements such as “I felt safe” or “adults listened” may hold a dif-
ferent meaning or significance for a 10-year-old than for a 17-year-old.
The YPQ survey, as a self-report measure, suffers the same limitations as
other self-report methodologies. Participants may have been motivated by
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 665
social desirability to represent their youth program experiences more posi-
tively than they actually had been. Because most participants were surveyed
toward the end of their program experiences, responses may more accurately
reflect the later phases rather than the earlier phases of their experiences.
Because surveys were administered in groups, unknown biases associated
with the presence of other youth participants may have been present.
Random sampling of target and comparison groups across a greater diversity
of youth programs and over an extended time (e.g., annual assessment of
4-H experience) might yield more generalizable results. However, cumulative
perceptions are also subject to biases such as forgetting and over- or under-
generalization from experiences.
Future research should consider assessing the YPQ along with other mea-
sures to allow for testing of concurrent validity and the selection of contrasting
youth samples to permit assessment of predictive validity. In addition, a two-
step process for generating and testing quality factors using confirmatory as
well as exploratory factor analysis might yield richer, more authentic results.
The YPQ survey shows promise as a practice, research, and policy tool.
Recommendations for further development and use in those areas are offered
In practice, the YPQ appears to be a valid and reliable overall tool for
youth, ages 10–17, to give voice to that which they know best: their experi-
ences in a youth development program. The one-page survey may be
useful to gain rapid feedback from participants, particularly with use of
clickers or online surveys, regarding “how the program works for them.”
Findings of this study suggest that program leaders might expect different
patterns of response for younger and older youth. Leaders may follow up
with a qualitative discussion to gain a deeper, broader view of those expe-
riences. Engagement of youth voice enhances youth leadership (Larson
et al., 2006) and program improvement (Yohalem & Wilson-Alhstrom,
2010). Comparing youth views with staff or expert observations may yield
insights about program conditions or processes. Thus, development of sim-
ilar surveys or observation rubrics for program staff or expert evaluators,
volunteers, and parents would help provide a broader picture of program
quality. Evidence from YPQ cannot be construed as definitive evidence for
specified program outcomes. However, as a part of a formative evaluation
including other views on quality and tentative outcomes, participant feed-
back is a critical element for youth leaders to determine whether the pro-
gram is working as intended.
Downloaded by [] at 08:17 22 September 2015
666 B. Silliman and W. R. Schumm
As a planning tool, the YPQ provides a useful point of reference for
framing content and implementing curricula according to youth develop-
ment best practices. As a training tool, the YPQ has been used to help staff
understand key elements of program quality and to discuss scenarios in
which the NRC-IOM model and results of this research apply (Silliman &
Shutt, 2010). As an evaluation tool, YPQ input speaks to common qualities
of youth programs that may need to be supplemented by other tools to
examine experiences with more specialized components such as science
skills or leadership norms (Yohalem & Wilson-Ahlstrom, 2010).
Tools such as the YPQ are well adapted to process use (Sabatelli etal.,
2009) to inform program improvement, staff training, and marketing or educa-
tion with parents of prospective or active youth program participants, program
sponsors, or the community at large. In an environment of accountability and
comparative value, programs that demonstrate positive participant experi-
ences, especially when corroborated by external evaluations indicating quality,
distinguish themselves as a developmental asset and worthwhile investment.
This research answered several questions about the validity and reliability of
YPQ while inviting additional questions for study. Although the NRC model
was largely affirmed by factor analysis with teen and preteen samples, differ-
ences with the model suggest intriguing conceptual issues for further research:
How well do context features predict diverse program outcomes? How are
these qualities correlated? Are such qualities distinguishable in an integrated
or holistic program? How well can teens or preteens perceive and distinguish
these qualities? Are these the same qualities perceived by program staff or
expert observers? Programmatic issues point to a different set of questions:
How do quality elements, or perceptions of them, vary across programs of
diverse intensity, frequency, duration, setting, audience, and format? How
should negative experiences be examined or factored into the description of
program qualities? How does use of YPQ enhance youth voice relative to
program engagement, perceptions of program quality, or outcome reports?
An equally intriguing set of methodological questions might be explored,
such as How can qualitative descriptions yield additional insight about teens’
and pre-teens’ experiences? How might items be improved or NRC-based
question sets be refined? Do other formats such as observations, games, or
focus groups provide more accurate or less intrusive approaches?
Several evaluation issues present interesting research questions: How can
the YPQ be used most effectively in evaluation planning, training, implemen-
tation, and formative or summative evaluation? How should youth and others’
perceptions of program quality be integrated or weighted? How can the YPQ
contribute to program improvement? How can evidence standards of feasibil-
ity, propriety, accuracy, utility, and accountability (Yarborough, Shultha,
Hopson, & Caruthers, 2011) be applied to measurement of program quality?
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 667
Future research on the YPQ should address a wider audience, such as
youth from a broader range of ethnic and racial groups, youth participating
with organizations other than 4-H or programs of extended duration, or
youth programs other than those nested in a positive youth development
approach. Comparisons of youth responses by gender, ethnicity, or program
experience could benefit those interpreting as well as those developing pro-
grams. In addition, a thorough meta-analysis of research and practice across
age groups and venues (e.g., afterschool, youth clubs, camps) would help
update and integrate findings across the field.
Each step of progress in measuring program quality reminds practitio-
ners of the importance of quality programs for credible outcomes. Youth
organizations, funding agencies, and their range of stakeholders depend on
quality programs to produce valued results, yet program quality is more
often assumed than assessed. Quality assessment can corroborate quality
outcomes and enrich training, program development and implementation,
marketing and recruitment, and accountability to stakeholders. The YPQ
survey provides a simple yet effective tool that these organizations might put
to work to measure what they value. Finally, youth and family professionals
may want to consider a similar process of evaluating experiences of program
participants, including process use as debriefing with participants at the
close of a program or with staff immediately after the program.
Development of a reliable, inexpensive, and easy-to-administer survey could
help local programs and extended youth organizations monitor program
quality and use data to plan, improve, and document benefits. Staff commit-
ted to positive youth development philosophy and practice have used the
tool to train staff, engage youth, and report on programs (Silliman & Shutt,
2010). Implementation of such an instrument across events, as annual program
assessments, and including parallel staff, parent, and/or expert assessments
might provide valuable cumulative and comparative data. Given current
practice, the greater challenge may be to gather program quality data and to
seek youth voice (Yohalem & Wilson-Alhstrom, 2010). Nevertheless, every
practical step that enables positive youth development can contribute to rais-
ing the expectations and effectiveness of youth organizations, funders, and
regulators who shape youth program policy.
Camino, L., & Zeldin, S. (2002). From periphery to center: Pathways for youth civic
engagement in the day-to-day life of communities. Applied Developmental
Science, 6, 213–220.
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A concep-
tual framework or implementation fidelity. Implementation, 2, 40–49.
Downloaded by [] at 08:17 22 September 2015
668 B. Silliman and W. R. Schumm
Catalano, R. F., Berglund, M. L., Ryan, J. A. M., Lonczak, H. S., & Hawkins, J. D. (2002).
Positive youth development in the United States: Research findings on evaluations
of positive youth development programs. Prevention & Treatment, 5. Retrieved
Cicchetti, D., & Rogosch, F. A. (2002). A developmental psychopathology perspective
on adolescence. Journal of Consulting and Clinical Psychology, 70, 6–20.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests.
Psychometrika, 16, 297–334.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: Review of research on
the influence of implementation on program outcomes and the factors affecting
implementation. American Journal of Community Psychology, 41, 327–350.
Durlak, J. A., Weissberg, R. P., & Padan, M. (2010). A meta-analysis of after-school
programs that seek to promote personal and social skills in children and ado-
lescents. American Journal of Community Psychology, 45, 294–309.
Eccles, J. S., & Templeton, J. (2000). Community-based programs for youth: Lessons
learned from general developmental research and from experimental and
quasi-experimental evaluations. (Urban Seminar Series, Urban Health Initiative).
Cambridge, MA: Harvard University.
Edna McConnell Clark Foundation. (2012). How to assess organizational capacity.
Retrieved from
Friedman, A. (Ed.) (2008). Framework for evaluating impacts of informal science
education projects. Retrieved from
Gambone, M. A., Klem, A. M., & Connell, J. P. (2002). Finding out what matters for
youth: testing key links in a community action framework for youth develop-
ment. Island Heights, NJ: Youth Development Strategies, Inc.
Goldberg, L. R., & Velicer, W. F. (2006). Principles of exploratory factor analysis. In
S. Strack (Ed.), Differentiating normal and abnormal personality (2nd ed., pp.
209–237). New York, NY: Springer.
Granger, R., Durlak, J. A., Yohalem, N., & Reisner, E. (2007). Improving after-school
program quality. New York, NY: William T. Grant Foundation. Retrieved from
Hair, J. F., Jr., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate
data analysis (6th ed.). Upper Saddle River, NJ: Pearson/Prentice Hall.
Hamilton, S. F., Hamilton, M. A., & Pittman, K. (2004). Principles for youth develop-
ment. In S. F. Hamilton & M. A. Hamilton (Eds.), The youth development hand-
book: Coming of age in American communities (pp. 3–22). Thousand Oaks, CA:
Henderson, K. A., Bialeschki, M. D., Thurber, C., Schueler, W. L., & Marsh, P. (2007).
Components of camp experiences for positive youth development. Journal of
Youth Development, 3. Retrieved from
High/Scope. (2012). Youth Program Quality Assessment (YPQA). Retrieved from
Huang, D., & Dietel, R. (2011). Making afterschool programs better. (CRESST Policy
Brief). Los Angeles, CA: University of California. Retrieved from www.cse.ucla.
Kahn, B. A., Bronte-Tinkew, J., & Theokas, C. (2008). How can I assess the quality of
my program? Tools for out-of-school time practitioners. Research-to-Results Brief
2008–10. Washington, DC: Child Trends. Retrieved from
Downloaded by [] at 08:17 22 September 2015
Youth Program Quality Survey 669
Larson, R. (2000). Toward a psychology of positive youth development. American
Psychologist, 55, 170–183.
Larson, R., Walker, K., & Pearce, N. (2004). A comparison of youth-driven and adult-
driven youth programs: Balancing inputs from youth and adults. Journal of
Community Psychology, 33, 57–74.
Larson, R. W., Hanson, D. M., & Moneta, G. (2006). Differing profiles of developmen-
tal experiences across types of organized youth activities. Developmental
Psychology, 42, 849–863.
Larson, R. W., & Walker, K. C. (2009). Dilemmas of practice: Challenges to program
quality encountered by youth program leaders. American Journal of Community
Psychology, 45, 338–349.
Lefleur, J., Milke, M., & Sanzone, J. (2010). 4-H Science, Engineering, and Technology
(SET): Youth engagement, attitudes, and knowledge study. Washington, DC:
Policy Studies Associates. Retrieved from
Lerner, R., Lerner, J. A., and Colleagues. (2011). Positive development of youth: Report
of findings of the first seven years of the 4-H Study of Youth Development. Medford,
MA: Tufts University Institute for Applied Research on Youth Development.
Lerner, R. M. (2004). The good teen: Recruiting adolescents from the myths of storm
and stress. New York, NY: The Crown Publishing Group.
Lerner, R. M. (2009). The positive youth development perspective: Theoretical and
empirical bases of a strength-based approach to adolescent development. In C.
R. Snyder and S. J. Lopez (Eds.), Oxford handbook of positive psychology (2nd
ed., pp. 149–163). Oxford, UK: Oxford University Press.
Lerner, R. M., Lerner, J. V., Almerigi, J., Theokas, C., Phelps, E., Gestsdottir, S., … von
Eye, A. (2005). Positive youth development, participation in community youth
development programs, and community contributions of fifth grade adoles-
cents: Findings from the first wave of the 4-H Study of Positive Youth
Development. Journal of Early Adolescence, 25, 17–71.
Mahoney, J. L., Larson, R. W., Eccles, J., & Lord, H. (2007). Organized activities as
developmental contexts for children and adolescents. In J. L. Mahoney, R. W.
Larson, & J. S. Eccles (Eds.), Organized activities as contexts of development
(pp. 3–22). New York, NY: Psychology Press.
Michelson, E., Zaff, J. F., & Hair, E. C. (2002). Civic Engagement programs and youth
development: A synthesis. Washington, DC: Child Trends. Retrieved from www.
Mihalic S. (2004). The importance of implementation fidelity. Emotional and
Behavioral Disorders in Youth, 4, 83–86, 99–105.
Moore, K. A., & Hamilton, K. (2010). How out of school time program quality is
related to adolescent outcomes. Research Brief. Publication 2010-19. Washington,
DC: Child Trends.
National 4-H Headquarters. (2012). About 4-H: History. Retrieved from
National Research Council (NRC) and Institute of Medicine (IOM). (2002). Community
programs to promote youth development. In J. Eccles & J. A. Gootman (Eds.),
Board on Children, Youth and Families, Division of Behavioral and Social
Sciences and Education. Washington, DC: National Academy Press.
Norusis, M. (2006). SPSS 14.0 guide to data analysis. Upper Saddle River, NJ: Prentice
Downloaded by [] at 08:17 22 September 2015
670 B. Silliman and W. R. Schumm
Osborne, J. W., & Costello, A. B. (2004). Sample size and subject to item ratio in
principal components analysis. Practical Assessment, Research, and Evaluation,
9. Retrieved from
Peterson, W. (2001). National 4-H impact assessment project. Retrieved from www.
Ramey, H., & Rose-Krasnor, L. (2012). Contexts of structured youth activities and
positive youth development. Child Development Perspectives, 6, 85–91.
Reisner, E. R., White, R. N., Russell, C. A., & Birmingham, J. (2004). Building quality,
scale, and effectiveness in afterschool programs. A summary of the TASC study.
Policy Studies Associates. Retrieved from
Roth, J. K., & Brooks-Gunn, J. (2003). What exactly is a youth development program?
Answers from research and practice. Applied Developmental Science, 7, 94–111.
Roth, J. L., Malone, L. M., & Brooks-Gunn, J. (2010). Does the amount of participa-
tion in afterschool programs relate to developmental outcomes? A review of the
literature. American Journal of Community Psychology, 45, 310–324.
Sabatelli, R. M., Anderson, S. A., Kosutic, I., Sanderson, J., & Rubenfeld, S. (2009). A
validation study of the youth assessment device. Family Relations, 58, 361–372.
Scales, P. C., & Leffert, N. (2004). Developmental assets: A synthesis of the scientific
research (2nd ed.). Minneapolis, MN: Search Institute.
Scanlin, M. (2006). Inspirations: Developmental supports and opportunities of youths’
experiences at camp. Martinsville, IN: American Camp Association.
Schwartz, S. J., Pantin, H., Coatsworth, J. D., & Szapocznik, J. (2007). Addressing the
challenges and opportunities for today’s youth: toward an integrative model
and its implications for research and intervention. Journal of Primary Prevention,
28, 117–144.
Silliman, B. (2008). Taking Congress home: Effects of NC 4-H Congress on youth
behavior and interaction. Journal of Youth Development, 3, Article 0803PA001.
Silliman, B., & Shutt, R.E. (2010). Weaving evaluation into the fabric of youth devel-
opment. Journal of Youth Development, 5, Article 100503FA003.
Singletary, L., Smith, M., & Evans, W. P. (2006). Self-perceived 4-H leader competen-
cies and their relation to the skills youth learn through 4-H youth development
programs. Journal of Extension, 44, Article 4RIB2.
Theokas, C., Almergi, J. B., Lerner, R. M., Dowling, E. M., Benson, P. L., Scales, P. C.,
& von Eye, A. (2005). Conceptualizing and modeling individual and ecological
asset components of thriving in early adolescence. Journal of Early Adolescence,
25, 113–143.
Tiffany, J. S., Exner-Cortens. D., & Eckenrode, J. (2012). A new measure for assessing
youth program participation. Journal of Community Psychology, 40, 277–291.
Urban, J. B. (2008). Components and characteristics of youth development pro-
grams: The voices of youth-serving policy-makers, practitioners, researchers,
and youth. Applied Developmental Science, 12, 128–139.
Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The pro-
gram evaluation standards: A guide for evaluators and evaluation users (3rd
ed.). Thousand Oaks, CA: Sage.
Yohalem, N., & Wilson-Ahlstrom, A. (2010). Inside the black box: Assessing and
improving quality in youth programs. American Journal of Community
Psychology, 45, 350–357.
Zeller, R. A., & Carmines, E. G. (1980). Measurement in the social sciences: the link
between theory and action. New York, NY: Cambridge University Press.
Downloaded by [] at 08:17 22 September 2015
... While qualitative methods such as a case study or a phenomenological study enable the study of a phenomenon in-depth [16], observational measures of program quality allow for the description of behavior in natural environments [17,18]. In youth sport research, most studies have relied on self-report measures [19,20], and this study used the same quantitative self-reported measures from the youth players who participated in the youth volleyball project studied. These measures were selected because they help us to understand if the youth volleyball program-that was the focus of this research-provided a climate that promotes PYD based on the participating youths' perceptions [2]. ...
... In the present study, developmental outcome refers to the psychosocial developments attained by the athlete and further developments in sports as a result of youth involvement in organized sports [31]. The growing number of sport-based youth development programs provide a potential avenue for integrating sport meaningfully into a development agenda [19]. As a result, program quality has been outlined as one of the predictors of the developmental outcomes resulting from participation in youth programs [12]. ...
... Additionally, the role that the sport experience plays in the development of positive personal and life skills in youth is also recognized [38,39]. In the current study, the authors used a hybrid of quality measures, including youth volleyball participants' perceived quality of talent identification and the development of a system for recognizing talent, the extent of participation in youth volleyball, an integrated system for youth volleyball, and national and regional volleyball competitions [15,19]. In a complex setting such as youth volleyball, different program qualities interact to produce positive outcomes for the youth [7]. ...
Full-text available
This study aims to assess program quality and developmental outcomes of a youth volleyball project in one of the regional states in Ethiopia, and further examine variations between groups across gender and project site zones. We applied a cross-sectional survey design, collecting quantitative data from youth volleyball players (n = 215) with a mean age of 16.18 years (SD = 0.69) through a self-reported questionnaire. The results indicated that young players' perceptions did not vary significantly across gender, except for the mean score of the perceived experience variable for girls (M = 2.68, SD = 0.318) was significantly higher than the mean score of boys (M = 2.58, SD = 0.258). One-way (project site zone) analyses of variance (ANOVAs) identified that youth volleyball projects in the central zone were consistently rated higher than those in the western zone, except for the current practice rating. Moreover, correlation analysis results indicated the presence of a significant relationship, both within and between program quality and developmental outcome variables. Furthermore, the results of regression analysis indicated that the program quality variables together predicted each of the developmental outcomes, accounting for 18.9% to 31.7% of the variances. It is concluded that the quality of the youth volleyball program in Ethiopia varies considerably across the project site zones and the program quality variables significantly relate to the developmental outcomes measured with differential effects. The data from this study reveals several practical applications for Ethiopia and beyond in terms of guiding youth volleyball projects. Moreover, the findings of the study showed that youth sport and the manner in which it is structured and delivered to youth players influences the attainment of positive developmental outcomes. These results suggest that contextual differences really do have an effect on the quality of youth sport program processes and developmental outcomes.
... The youth athletes completed the demographic form and the Youth Program Quality Survey (Silliman & Schumm, 2013) for the 1st data collection time point. In the 2nd data collection, youth athletes were asked to participate in focus Content courtesy of Springer Nature, terms of use apply. ...
... To measure the youth athletes' experiences in the soccer program, researchers used the Youth Program Quality (YPQ) survey (Silliman & Schumm, 2013), which was designed to cover the eight contextual features that support positive youth development, as identified by the National Research Council (NRC) and Institute of Medicine. This 24-item survey has participants rate items on a 4-point scale (1 = "strongly disagree" to 4 "strongly agree"). ...
... This 24-item survey has participants rate items on a 4-point scale (1 = "strongly disagree" to 4 "strongly agree"). Silliman and Schumm (2013) established content validity, as well as construct validity through an exploratory factor analysis on a sample of youth 10-13 years old and 14-17 years old. ...
Full-text available
Research connects organized sport participation with greater psychological and social benefits in youth athletes. In order to best promote these positive outcomes, youth sport programs must be active in their implementation of effective models that promote positive youth development. This study explored the implementation of the Five C’s model in a youth academy soccer program. This longitudinal study explored the perceptions of youth athletes and members of their ecological system (e.g., coaches and parents). Using a mixed method approach, 145 youth athletes, 128 parents, and 23 coaches responded to surveys and/or participated in focus groups. Survey results indicate the implementation of the Five C’s led to more opportunities for positive development among youth athletes, an increase in the positive affects youth athletes experienced throughout the program, an increase in parental support of the academy, and a shift in coaching philosophies that promote the Five C’s. Together these results indicate change can be made when youth sport organizations implement positive youth development models. Additionally, these models could serve as a foundation for improving a long-lasting commitment to youth sport.
... An adapted version of the Youth Program Quality Survey (YPQS) was used to assess convergent validity to attain youth's perceptions of program quality (Bean & Forneris, 2016b;Silliman & Schumm, 2013). Such a measure was selected because it is also based on the NRCIM's eight setting features (Eccles & Gootman, 2002). ...
... It was important to understand if the observed assessment of program quality (measured by the PQAYS) was congruent with youth's perceptions of their program experiences. As the YPQS is relatively new, few studies have utilized this measure; however, past findings have revealed moderate to high instrument reliability (α = .60-.96; Silliman, 2008;Silliman & Schumm, 2013;Silliman & Shutt, 2010). ...
Full-text available
Research has demonstrated that quality sport programs have the potential to foster the physical and psychosocial development of youth. However, there is an absence of observational measures to assess program quality related to psychosocial development within youth sport. The purpose of this paper is to report on two studies conducted to develop a valid and reliable observational measure to assess program quality processes in youth sport. Study one outlines the process of attaining content and face validity using an expert panel approach when developing the Program Quality Assessment in Youth Sport (PQAYS) observational measure through a review of literature and collaboration with expert academics and coaches. Study two outlines further steps taken to test the internal reliability, as well as convergent and predictive validity of the measure. Results from the two studies provide initial evidence that the PQAYS is a valid and reliable measure that can be used in intervention and evaluation research within youth sport. © 2018, © 2018 The Author(s). This open access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license.
... Gainera, bizipen txarren aldagaietan goi-mailako programetan balore baxuagoak aurkitu zituzten behe-mailakoetan baino (Bean, Harlow, et al., 2018). 138 aztertu ondoren kasu bietan bai kalitatea bai oinarrizko beharrak asebetetzen zirela aurkitu zuten, Youth Program Quality Survey (YPQS) (Silliman & Schumm, 2013;Silliman & Shutt, 2010) eta Learning Climate Questionnaire (LCQ) (Standage & Duda, 2005) galdetegien bitartez, hurrenez hurren. Programa hauetan nerabeek (14±2 urte; %80 kaukasoarra) taldeko jarduera fisiko eta kiroletan, zehazki boleibola, kale-hockeya eta izotz-hockeyan parte hartu zuten. ...
... The popularity of language workshops might be a result of the limited access to foreign/second languages for those underprivileged students, and language workshops therefore addressed their long-term needs. There are also possible explanations for students' favorable attitude towards volunteer/educational camps, as research has documented its capability in promoting socialization and improving emotional climate (Silliman and Schumm 2013). ...
Full-text available
This study validated a subjective outcome evaluation scale based on the perceptions of service recipients and examined the normative profiles as well as correlates of client satisfaction under a corporate-university-community Service-Learning (SL) programs, where university students conducted the SL programs with high school students as the service recipients. Based on data collected over three years from high school service recipients (N = 1854) who responded to a subjective outcome evaluation measure which assesses perceived program content, program implementers and benefits, exploratory and confirmatory factor analyses supported the three-factor model, and the three subscales showed good factorial validity, convergent validity, concurrent validity and internal consistency. Most of the service recipients had positive perceptions of the different aspects of the program. Regarding correlates of client satisfaction, the three aspects of satisfaction all had predictive effects on the overall client satisfaction. Client satisfaction ratings also differed significantly across different activities and grades.
... An adapted version of the Youth Program Quality Survey (YPQS) was used to assess PQ at T1 of the study (Bean & Forneris, 2016a;Silliman & Schumm, 2013). This measure is comprised of 19-items within four subscales including: (a) appropriate adult support and structure (5 items, e.g. ...
Acknowledging the importance of longitudinal data to test process-based psychological theories of motivation is critical. The purpose of this study was to use a person-centred approach to identify unique subgroups (i.e. profiles) of youth athletes based on their level of self-reported programme quality (PQ) and basic needs support mid-way through their sport season and investigate potential differences between the subgroups on their self-reported basic needs satisfaction at the end of the sport season. The current study involved 541 Canadian youth athletes (males n = 289; females n = 250; gender-fluid n = 2) within 52 sport programmes over the course of 18 months. Youth athletes ranged in age from 8 through 19 (M = 13.76, SD = 2.61). A latent profile analysis (LPA) in Mplus 8.0 was used to carry out the analyses. The LPA revealed three distinct profiles based on youth athletes’ levels of self-reported PQ and basic needs support. Specifically, athletes who perceived their sport experience to be of higher quality and supported their basic psychological needs midway through the sport season also reported higher levels of basic needs satisfaction at season end. Results from this study contribute to the field of sport psychology through understanding how basic needs theory contributes to the dimensions of programme quality and by informing recommendations for future coach education on how to satisfy youth athletes’ basic psychological needs for autonomy, competence, and relatedness through programme delivery.
Researchers have documented positive associations among youth program quality and academic outcomes, primarily based on cross-sectional data. This study examined longitudinal associations among youth-reported program experiences and academic expectations, self-reported grades, and perceived value of school using data from the national evaluation of Boys and Girls Clubs of America (BGCA). The sample included 101,050 Club attendees at 2,741 BGCA sites throughout the United States from 2015-2018. Latent Growth Curve Modeling was used to examine change in youth-reported program experiences as well as the longitudinal associations among perceived program experiences and academic outcomes over time. Baseline perceptions of program experiences were associated prospectively with increased perceived value of school. In addition, gains in youth-reported program experiences predicted gains in each of the academic outcomes. These findings suggest that youth programs can promote positive academic trajectories when youth perceive the programs as continuing to meet their needs over time.
Youth sport programme structures and processes can directly influence participation outcomes, including mental health and well-being. Researchers have found that programme quality and basic needs satisfaction foster psychosocial outcomes; however, limited research has examined the mechanisms that facilitate mental health. The purpose of this study was to examine relationships between programme quality, basic needs satisfaction, and mental well-being in competitive youth sport. Youth (N = 160, 89 males, 71 females; Mage = 15.36, SD = 2.50) completed questionnaires at mid-season, with questions related to quality of their sport programming, and at programme end, with questions pertaining to basic needs satisfaction and mental well-being. Structural equation modelling was used to test the direct relationships between programme quality and mental well-being (r = .46, p < .001; model 1) and between basic needs satisfaction and mental well-being (r = .63, p < .001; model 2). Using several fit indices, results showed an adequate fit of both models to the data, suggesting that programme quality significantly predicted basic needs satisfaction and basic needs satisfaction significantly predicted mental well-being. Bootstrapping analysis was used to test if basic needs satisfaction mediated the relationship between programme quality and mental well-being. Results supported mediation, with a large effect (k² = 0.28). The study findings help to emphasise the value of structuring youth sport programmes to satisfy basic needs, which may have positive implications on youth mental health. This is important because supporting mental well-being at an earlier age may facilitate mental health and well-being in adulthood.
Full-text available
This article analyzes the development of initiative as an exemplar of one of many learning experiences that should be studied as part of positive youth development. The capacity for initiative is essential for adults in our society and will become more important in the 21st century, yet adolescents have few opportunities to learn it. Their typical experiences during schoolwork and unstructured leisure do not reflect conditions for learning initiative. The context best suited to the development of initiative appears to be that of structured voluntary activities, such as sports, arts, and participation in organizations, in which youths experience the rare combination of intrinsic motivation in combination with deep attention. An incomplete body of outcome research suggests that such activities are associated with positive development, but the developmental processes involved are only beginning to be understood. One promising approach has recorded language use and has found that adolescents participating in effective organizations acquire a new operating language that appears to correspond to the development of initiative.
Full-text available
An Empowerment Evaluation process served to engage staff and campers and foster integration of authentic assessment methods into youth camp programming over a three-year period. Key elements to the process included program planning, staff training, timing and balance of action and reflection activities, data collection and management. Camp staff involved for 2-3 years reported improved focus and staff who served in the third year achieved mastery in communication, leadership, teaching, and management skills. A purposive sample of three different camp venues evaluated in Year 3 indicated that campers improved significantly in outdoor and life skills. Over 70% felt safe, supported, and enabled to build skills. Implications for practice, research, and policy are discussed.
Full-text available
This evaluation report describes the outcomes and quality of the 2006 North Carolina 4-H Congress, an annual five-day teen conference focused on citizenship, leadership, and service. A majority of returning youth cited Congress experiences as significant in their continued learning and practice in citizenship, leadership, and service learning. Likewise, most youth participants in the 2006 conference indicated that they planned to participate in more citizenship, community leadership, and service activities in their home communities. A Youth Program Climate survey revealed that youth viewed NC 4-H Congress as a setting where service was important, where they learned to accept differences, teamwork was emphasized, and where they were able to make a difference in the lives of others. Three implications of the evaluation report are discussed: 1) value of a youth leadership conference for educating and inspiring youth in citizenship, leadership, and service; 2) evaluation methodology, including engaging youth leaders in design and use of conference data; and 3) marketing and accountability opportunities resulting from program evaluations.
Full-text available
Implementation fidelity refers to the degree to which an intervention or programme is delivered as intended. Only by understanding and measuring whether an intervention has been implemented with fidelity can researchers and practitioners gain a better understanding of how and why an intervention works, and the extent to which outcomes can be improved. DISCUSSION: The authors undertook a critical review of existing conceptualisations of implementation fidelity and developed a new conceptual framework for understanding and measuring the process. The resulting theoretical framework requires testing by empirical research. SUMMARY: Implementation fidelity is an important source of variation affecting the credibility and utility of research. The conceptual framework presented here offers a means for measuring this variable and understanding its place in the process of intervention implementation.
Full-text available
This article reports the results of a statewide survey to assess the influence of perceived 4-H volunteer leader skills on the life skills 4-H youth learn. Results indicate the most important skill a volunteer leader possesses is to ensure the physical and psychological safety of 4-H members. This includes keeping youth from hurting each other's feelings; keeping youth from bullying each other; managing conflict between youth; making sure that the facility where 4-H youth meet is safe. These results emphasize the importance of the careful recruitment, screening, training, and management of 4-H volunteer leaders.
Full-text available
Statisticians have wrestled with the question of sample size in exploratory factor analysis and principal component analysis for decades, some looking at total N, some at the ratio of subjects to items. Although many articles attempt to examine this issue, few examine both possibilities comprehensively enough to be definitive. This study examines a previously published data set to examine whether N or subject to item ratio is more important in predicting important outcomes in PCA. The results indicate an interaction between the two, where the best outcomes occur in analyses where large Ns and high ratios are present.
The Oxford Handbook of Positive Psychology studies the burgeoning field of positive psychology, which, in recent years, has transcended academia to capture the imagination of the general public. The book provides a roadmap for the psychology needed by the majority of the population-those who don't need treatment, but want to achieve the lives to which they aspire. The articles summarize all of the relevant literature in the field, and each is essentially defining a lifetime of research. The content's breadth and depth provide a cross-disciplinary look at positive psychology from diverse fields and all branches of psychology, including social, clinical, personality, counseling, school, and developmental psychology. Topics include not only happiness-which has been perhaps misrepresented in the popular media as the entirety of the field-but also hope, strengths, positive emotions, life longings, creativity, emotional creativity, courage, and more, plus guidelines for applying what has worked for people across time and cultures.
Designed to bridge the gap between theorists and methodologists by presenting an integrated approach to measurement. It differentiates between random and systematic error thus conveying both statistical techniques and their theoretical underpinnings. The book is written at a level accessible to students of social sciences with some statistical training, and does not assume a sophisticated mathematical background. Chapters include factor analysis, reliability, validity and evaluating systematic error.-K.Clayton
This article summarizes a much lengthier one that appeared in Prevention and Treatment. The earlier article grew out of a project initiated by the Department of Health and Human Services Office of the Assistant Secretary for Planning and Evaluation. The Positive Youth Development Evaluation project described why policy makers, practitioners, and prevention scientists advocated a shift in approach for how youth issues are addressed in this country. The Positive Youth Development Evaluation project sought to define how youth development programs have been defined in the literature and then to locate, through a structured search, strong evaluations of these programs and summarize the outcomes of these evaluations. In the current article, we explain why prevention has shifted from a single problem focus to a focus on factors that affect both positive and problem youth development, describe what is meant by positive youth development, and summarize what we know about the effectiveness of positive youth develop...
What are the types of environments in which youth thrive? How do we cultivate such environments to promote optimal development and positive behavior in youth? The Youth Development Handbook: Coming of Age in American Communities provides youth and development practitioners access to current theory and research in the field of youth development, including illustrations of good practice, original case studies, and a contextual approach to such topics as youth participation and diversity. The Youth Development Handbook is designed for scholars and researchers in applied developmental science as well as practitioners and policy makers who implement youth development initiatives. The book is also recommended for use in graduate courses on youth development in the fields of Psychology, Human Development & Family Studies, and Education.