Content uploaded by Kevin Dadaczynski
Author content
All content in this area was uploaded by Kevin Dadaczynski
Content may be subject to copyright.
euPrevent quality checklist
A checklist to assess the quality of community based health
promoting programs
Authors
Kevin Dadaczynski, M.Sc. 1
Silvia de Ruiter, M.Sc. 2
Goof Buijs, M.Sc. 2
Nanne de Vries, Ph.D 3
1 Leuphana University, Germany
2 Netherlands Institute for Health Promotion NIGZ
3 Maastricht University, the Netherlands
Publication date
February 2012
Financed by
euPrevent
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
2
Contents
Introduction .................................................................................................................................... 3
Aim and target group of the euPrevent quality checklist ........................................................ 3
Preamble ....................................................................................................................................... 4
The quality model ........................................................................................................................... 5
euPrevent quality checklist ............................................................................................................ 7
Appendix 1 Glossary of basic terms ............................................................................................ 25
References ..................................................................................................................................... 29
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
3
Introduction
The euPrevent quality checklist can be used to assess the quality of community based, health
promoting programs for children and adolescents using quality indicators.
In this document the term program is defined as “planned actions, which are based in health
sciences and aim at systematic and sustainable changes of individual behaviour and/or the
environment. Health programs can aim at the promotion of health/ well-being (health promotion)
or at the avoidance of diseases (prevention)” (1).
Aim and target group of the euPrevent quality checklist
The checklist aims to support stakeholders working in community based health promotion to
promote quality and transparency of health promoting programs in communities in a structured
way.
The main target group of the checklist are national and regional stakeholders, i.e. governmental
organisations (e.g. ministries, municipalities), public health services, public organisations (e.g.
health insurances) and NGOs or private organisations (e.g. foundations, associations) working in the
area of health promotion and prevention and specifically in community based health promotion.
The checklist can support them to make a national or regional inventory of existing community
based programs available in the area of health promotion of children and adolescents.
The results of the quality assessment can support communities in selecting programs which fulfil
quality requirements and fit specific community needs. Also providers of already existing programs
and developers of new programs can benefit from the checklist, giving them a better understanding
of the quality components of a program and therefore how they can improve the quality of their
program.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
4
Preamble
The checklist is based on the euPrevent quality checklist that was developed to asses the quality of
community based programs on healthy eating and physical activity for children and adolescents. The
literature review therefore mainly focused on community based programs on healthy eating and
physical activity and not health promotion in general. However the authors agree that the checklist
can also be used to asses programs focused on health promotion in general.
For more information on how to make an inventory of existing community based programs the
authors refer to the euPrevent quality tool in which the process of making an inventory is described
step-by-step.
The euPrevent quality tools are an adaptation of the HEPS Inventory Tool that was developed as
part of the HEPS project (2008-2011). The HEPS project supports countries in Europe to promote
healthy eating and physical activity in schools in a positive and sustainable way (see
www.hepseurope.eu).
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
5
The quality model
The term quality is a broad concept with different perspectives. To assess the quality of community
based programs in its entire scope it is necessary to consider all relevant aspects of a program and
not just to focus on a specific part such as the achieved outcomes. According to Donebedian, quality
of programs should be assessed using three dimensions: quality of structure, quality of process and
quality of outcome (2).
The quality of structure is primarily related to the conditions under which the program is
provided. This includes for example the organisational structure, material resources such as
equipment, financial resources and human resources such as qualified personnel. Quality of
process refers to actions within the delivery phase of the program including support, monitoring,
attainability and acceptability of the target group. Quality of results includes the desirable and
undesirable changes in the target group and the setting. This usually includes aspects of
effectiveness and aspects of efficiency.
For health promotion and prevention these dimensions are often transferred and applied. It has
been proposed that the quality of program should also take other aspects into account such as
assessment and planning (3, 4). Since both aspects refer to the same phase of a program, the term
quality of concept is used here. This dimension refers to how the program is conceptually based
and includes amongst other things a needs assessment, a definition of target groups and objectives
as well as the development of program methods.
The HEPS quality model which is illustrated in Figure 1 builds on the four quality dimensions
described above. As a structured sequence it reflects a holistic view of quality. To assess the quality
of a health promoting program it is necessary to consider each quality dimension. As illustrated in
the model each dimension has an influence on the following one. Failures and mistakes, for example
in the planning phase, can lead to inconsistencies and cause difficulties in the process and
evaluation phase. If no or only unspecific aims were formulated in the planning phase, their
achievement is not assessed in the evaluation. Derived from the quality paradigm of optimisation, a
cyclic perspective is included in the HEPS quality model. This means that the results of the
comprehensive quality assessment should be used for continuous quality improvement.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
6
Figure 1: The HEPS quality model
Quality of
concept
Quality of
structure
Quality of
process
Quality of
results
theoretical basis
and planning of
a program
Assessment &
Analysis
Target group &
Objectives
Principles of
Health
Promotion
Intervention
Planning
conditions
under which the
program is
realized
Program
Management &
Structure
Dissemination
factors that
effect the
implementation
of the program
Implementation
& Delivery
Sustainability
results of the
program
Health
Outcomes &
Efficacy
Intermediate
Program
Outcomes
Overall
program
quality
Improvement
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
7
euPrevent quality checklist
A checklist to assess the quality of community based health promoting programs.
For further assistance have a look at the indicator description and at the glossary which can be
found in Appendix 1.
Title of the program:
Assessor
Name:
Organisation:
E-mail address:
Date of assessment:
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
8
Quality Dimension 1: Quality of concept (yes=2 points, partly= 1 point, no= 0 points)
1. Assessment & Analysis
Page
Score
1.1
Is the program based on a needs assessment of the target community?
1.2
Has a ‘community profile’ been developed which guided the planning of the
community based program?
1.3
Is the program planning and implementation based on an assessment of the
communities’ readiness for change?
2. Target group & Objectives
2.1
Is the target group clearly and specifically defined?
2.2
Are the objectives clearly and specifically described?
3. Principles of Health Promotion
3.1
Is the program based on a positive and comprehensive concept of health, which is
focused on the promotion of resources and skills to enhance health?
3.2
Have children and adolescents been actively involved in the planning of the
program?
3.3
Does the program apply behavioural and environmental strategies, which have
an impact on health?
3.4
Does the program consider different settings within the community (whole of
community, family/ home environment, and school) to promote health?
4. Program planning
4.1
Is the program based on a theory of change which uses a socio–ecological
understanding of health?
4.2
Are elements of social marketing and media communication embedded in the
program?
4.3
Have community members and relevant organisations been actively involved in
all stages of the program (planning, implementation, and evaluation)?
4.4
Are the specific conditions and processes of the implementation settings
sufficiently considered in the planning of the program?
4.5
Does the program fit with already existing local, regional or national policies and
strategies?
4.6
Does the program consider the different requirements of the target group (e.g.
disadvantaged groups and/or with regard to the social, cultural and
developmental background)?
4.7
Does the program use multiple strategies to promote health?
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
9
Overall score
Score
Assessment
Final Score
32 – 24
High quality of concept
2
23 – 12
Average quality of concept
1
11 – 0
Low quality of concept
0
Please circle the final score in the right column which matches with the reached score in the left
column.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
10
Quality Dimension 2: Quality of structure (yes=2 points, partly= 1 point, no= 0 points)
5. Program Management & Structure
Page
Score
5.1
Is an appropriate organisational structure described in the program?
5.2
Is a detailed implementation plan available which specifies how the program
activities are to be implemented?
5.3
Is a detailed description of the costs/budget available which provides
information on the financial resources needed to carry out the program?
5.4
Does the program provide training for those who implement the program?
6. Dissemination
6.1
Does the program provide easy access to relevant materials needed for program
implementation (e.g. through a website)?
6.2
Does the program establish partnerships with local, regional and national
organisations in terms of its dissemination?
Overall score
Score
Assessment
Final Score
12 – 9
High quality of concept
2
8 – 4
Average quality of concept
1
3 – 0
Low quality of concept
0
Please circle the final score in the right column which matches with the reached score in the left
column.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
11
Quality Dimension 3: Quality of process (yes=2 points, partly= 1 point, no= 0 points)
7. Implementation & delivery
Page
Score
7.1
Are the roles and responsibilities of organisations and persons who are involved
in the program clearly defined?
7.2
Are strategies described which ensure tailoring of the program activities to the
specific context in which they will be implemented?
7.3
Are the marketing messages of the program delivered in a positive and non-
stigmatizing way?
7.4
Are concrete strategies applied to reach and keep disadvantaged children and
adolescents involved?
7.5
Does the program provide a monitoring tool which allows an ongoing assessment
of the implementation?
7.6
Is ongoing support (e.g. counselling) provided to those delivering the program?
7.7
Does the program method fit with the principles of health promotion?
8. Sustainability
8.1
Does the program try to establish active links with other relevant initiatives in
the same community?
8.2
Does the program include capacity building on an individual, organisational and
environmental level?
Overall score
Score
Assessment
Final Score
18 – 13
12 – 6
5 – 0
High quality of concept
Average quality of concept
Low quality of concept
2
1
0
Please circle the final score in the right column which matches with the reached score in the left
column.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
12
Quality Dimension 4: Quality of results (yes=2 points, partly= 1 point, no= 0 points)
9. Health Outcomes & Efficacy
Page
Score
9.1
Are the evaluation methods and procedures based on agreed scientific
standards?
9.2
Do the program results demonstrate positive health outcomes?
9.3
Is there evidence that the positive health outcomes sustain over a period of at
least 6 months?
9.4
Is the overall effort (costs, time) in balance with the effectiveness of the
program?
10. Intermediate Program Outcomes
10.1
Does the program have a positive impact on capacity building on an individual,
organisational, and environmental level?
10.2
Has the program been successfully adopted by other communities (communities
that were not part of the development and evaluation)?
10.3
Have the results of the process evaluation been used to improve the program?
Overall score
Score
Assessment
Final Score
14 – 10
9 – 5
4 – 0
High quality of concept
Average quality of concept
Low quality of concept
2
1
0
Please circle the final score in the right column which matches with the reached score in the left
column.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
13
Summary
In the table below you can summarise the scorings of the program. Please fill in the achieved score
of each quality dimension in the middle column and a final rating in the right column, which
indicates a final assessment of each dimension (see table above). After summing up the final scores
the total value can be compared with the overall evaluation table.
Quality Dimension
Reached score
Final score
Quality of Concept
…………/ 32
Quality of Structure
…………/ 12
Quality of Process
…………/ 18
Quality of Outcomes
…………/ 14
Total
Total Score
Overall evaluation of program quality
8 – 6
High program quality
5 – 3
Average program quality
0 – 2
Low program quality
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
14
Description of indicators
Below you can read a short description for each indicator, which will assist you in using the quality
checklist. Please read these carefully before assessing each indicator. Specific terms which are
marked in italic are described in the glossary in more detail.
1. Quality of concept
1.1 Is the program based on a needs assessment of the target community?
Successful programs tailor their activities to the specific needs of the community. This
requires a comprehensive and systematic analysis of the problem, its distribution in the
community and its health related consequences. This also requires an analysis of factors
contributing to health related problems which facilitates program planning (5). One possible
tool that can be used is the Analysis Grid for Environment linked to Obesity (50, 51). This
tool which has been widely used to develop various community based programs categorizes
components of the unhealthy environment into two sizes of environments (micro/ settings
and macro/ sectors) and four types of environments: (1) the physical environment (what is
or is not available; e.g. availability of training opportunities, food outlets); (2) the economic
environment (the financial factors, e.g. costs of food production); (3) the political
environment (the rules, e.g. government food and nutrition policies on the macro-
environmental level); and (4) the socio-cultural environment (communities or societies
attitudes, beliefs, and values related to health).
1.2 Has a ‘community profile’ been developed which guided the planning of the community
based program?
A community profile helps program developers to better understand the structures and
conditions as well as the underlying cultural and social roots of the community which
contribute to health related problems. Knowing these factors, and also the available
resources of the community (e.g. available services, programs, opportunities for physical
activity), allow a program to be developed which fits with the specific requirements of the
community. It is important to not only focus on problems but also on the communities’
assets and resources as these can be used in program implementation. In developing the
community profile it is recommended to use local data which are already available (e.g. data
from the local health department or data from school screenings) (52). In addition it might
be useful to gather data from community residents to identify resources available, needs and
preferences.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
15
1.3 Is the program planning and implementation based on an assessment of the communities’
readiness for change?
Programs that ensure and support readiness for change are more likely to be successful than
programs which do not include this aspect. Two target groups need to be considered when
talking about readiness for change. Of course it is needed to motivate (1) the target group
itself (e.g. children and adolescents). Also it is important to (2) motivate all persons involved
in the implementation of the program (multiplier, stakeholder etc.). Everyone involved in the
program, needs to be prepared to invest in the prevention of health related problems in
children and adolescents. This requires a vision about the program and its benefits which
can be supported by strong leadership.
2.1 Is the target group clearly and specifically defined?
It must be made clear for which target group the program has been developed. The more
specifically the target group is defined the better. This makes it possible to evaluate whether
the target group was reached successfully. Inclusion criteria which specify the target group
should be stated such as age, gender and socio-economic background. When different target
groups are intended for the program each group needs to be specified. Exclusion criteria
such as age group or gender can also be stated. The HEPS quality checklist is oriented on
community based programs on the prevention of health related problems in children and
adolescents, not on treatment or secondary disorders.
2.2 Are the objectives clearly and specifically described?
It is important to be clear about expectations of the outcome of the program. The key in
program success lies in clear and specific objectives. An objective should be defined clearly
and specifically with little room for interpretation. “Specific” means that the objective has to
be defined by the measurable variables which the program aims to change. For example:
increase the number of boys and girls aged 8 to 10 years participating in the street dance
lessons at the community health centre.
Having specific and measurable objectives is a prerequisite to being able to assess the
degree of target achievement. Unspecific and unclear objectives do not allow for meaningful
evaluation and increase the chance that any change could be claimed a success for the
program. Objectives must be realistic and sensible. For example, it may be desirable to
reduce the number of overweight children and adolescents by 100%. But this is not realistic
and will depend on a great number of factors which cannot all be changed within the
community setting.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
16
3.1 Is the program based on a positive and comprehensive concept of health, which is focused
on the promotion of resources and skills to enhance health?
A positive concept of health focuses on the resources, skills and abilities of people which
enable them to cope with everyday demands and make healthy decisions. Programs which
are based on this positive health promotion concept improve the health by strengthening
skills and capabilities of individuals to enable them to take responsibility and increase
control over their life and health. Programs that are solely focused on the treatment of risks
and complaints by viewing the individual as a patient are not based on a positive concept of
health.
3.2 Have children and adolescents been actively involved in the planning of the program?
Community based prevention of health related problems requires a coordinated community
effort in which all persons and organisations who influence respectively are influenced by
the problem are involved (9). Therefore it is important to involve a representative group
(including those who are hard to reach) of children and adolescents in the planning of
programs that target them. Involving them as early as possible promotes ownership and
sustainability. Acting bottom-up by involving the community contributes to community
empowerment and promotes community moral which in turn enhances long-term
participation (10,11).
3.3 Does the program apply behavioural and environmental strategies, which have an impact on
health?
Programs are more likely to be effective if they focus on factors influencing individual
behaviours as well as looking at the environment (12). This approach takes into account
health related behaviours such as diet or smoking and their modification as well as related
health topics like coping with stress or self-esteem and self-efficacy. These apply directly on
the individual level.
With ‘environmental strategies’ we refer to the physical, economic and social environment
in the community, which each has an impact on individual health. These include measures
such as the creation of playgrounds or the reduction of junk food sold at tuck shops in the
neighbourhood.
3.4 Does the program consider different settings within the community (whole of community,
family/ home environment, and school) to promote health?
The health of children and adolescents is complex and influenced by individual, social/
cultural, environmental, and policy determinants. Therefore community based programs
need to aim at all settings which have an impact on children’s and adolescents’ health. This
does not mean that programs that are exclusively implemented in a single setting are not
successful. But focusing on a single setting can lead to inconsistent messages. Derived from
ecological models (13,14) it is known, that the family, the school (as Microsystems) but also
the community as a whole (as Exosystem) are very important for children and adolescents.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
17
However, research clearly shows that parents’ behaviour, parenting style and food
environments provided at home have a significant influence on the development of the
eating behaviours of children and adolescents (especially in the early years) (15,16). It is
therefore essential that parents (or caregivers) are included in community based programs.
4.1 Is the program based on a theory of change which uses a socio–ecological understanding of
health?
Successful programs are based on an intervention theory which provides an understanding
about the determinants of the health problem, clarifies how these factors are
interconnected, and gives guidance on how change can be induced (17). Given the fact, that
health is influenced by a complex interplay of individual, social/ cultural, environmental, and
policy determinants, a socio-ecological framework enables its users to consider the whole
system that surrounds the individual and affects individual behaviour. The environmental
changes can be described by using four categories (as part of the ANGELO framework):
physical, economic, policy, and socio cultural (see 1.1) (6).
4.2 Are elements of social marketing and media communication embedded in the program?
Media play a significant role in the lives of children and adolescents. In times of increased
confrontation but also of increased access, particularly children and adolescents are
interested in and susceptible for media messages. This can also be used in health promotion
and prevention programs. Marketing and communication strategies can be used to recruit
the target group but also to spread the health message of the program and to raise
awareness within the community. These strategies include e.g. regular newsletters, flyers,
websites, posters, articles in local newspapers, radio or TV. Also the development and
distribution of ‘promotion items’ (e.g. stickers or water bottles with a health message) can
be included.
4.3 Have community members and relevant organisations been actively involved in all stages of
the program (planning, implementation, and evaluation)?
Community based programs should involve community organisations and residents as early
as possible to promote ownership and sustainability. Acting bottom-up by involving the
community contributes to community empowerment and promotes community moral which
in turn enhances long-term participation (10,11). Community participation should take
place at each stage of the program from planning to evaluation. Also it needs to be ensured,
that not only adults but especially children and adolescents are sufficiently involved in the
program planning and implementation.
4.4 Are the specific conditions and processes of the implementation settings sufficiently
considered in the planning of the program?
Given that community based programs are implemented in multiple settings it should be
taken into account that each context within which the program intervenes has its own
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
18
routines, organisational practices, values and norms. Hence, each program must tailor its
activities to the specific context. Thus, each context needs its own implementation strategy.
The more the program fits its activities to the context, the better it can be implemented
which in turn contributes to its success (10). Programs that consider the specific conditions
and process of the different settings in its planning meet the demands of this indicator.
4.5 Does the program fit with already existing local, regional or national policies and strategies?
Health promotion and prevention programs that operate on a community level have a better
chance to be successful (in terms of acceptance, funding, implementation and sustainability)
when aligned on existing health policies and strategies on a local, regional and national level
(5). This requires an analysis of existing policies and strategies and the application of these
in the planning of community based programs. Programs that have considered existing
health policies and strategies in the planning of the program meet the demands of this
indicator.
4.6 Does the program consider the different requirements of the target group (e.g.
disadvantaged groups and/or with regard to the social, cultural and developmental
background)?
Research indicates that socially deprived children and adolescents (e.g. with low
socioeconomic status (SES), and migration background) are more often affected by health
related problems (18,19). Hence, community based programs should aim particularly on
this target group. The programs need to show that the content and applied methods are
sensible for the intended target group in terms of gender, age, socio-economic und
multicultural background. With regard to age, methods should be appropriate for different
physical and cognitive developmental stages. For example, young children have other
sporting interests and motoric skills than adolescents. In addition, family involvement is
more important in childhood than in adolescent age. With regard to multicultural issues, the
program should be sensitive. Eating habits and behaviours depend greatly on different
cultural norms and values. Programs that comprehensively and systematically consider
target group characteristics in the planning of the program content and methods fully meet
the demand of this indicator.
4.7 Does the program use multiple strategies to promote health?
Multi-strategy resp. multi-component programs are promising and should therefore be
applied for community based programs. Strategies applied by other programs include
capacity building, health skills, policy development and marketing and communication.
These programs not only focus on individual skill development. They also aim at a wide
array of measures, including partnering with stores and restaurants, local policies for safe
and active transport or the promotion of walking/ riding from and to school, playgrounds,
etc. (20-24). Some programs improved local parks, enhanced access to facilities and
developed school health policies (ibid.).
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
19
2. Quality of structure
5.1 Is an appropriate organisational structure described in the program?
The organisational structure of the program should be specified. This includes the
establishment of a steering committee and if relevant of additional groups such as an
advisory group which supports and monitors the work of the steering group. Moreover a
program manager or coordinator as well as project staff needs to be appointed. It has been
shown that leadership plays a crucial role in program success (4) (see 1.4). There is also a
need to define a functional structure which includes regulations about operational
procedures (e.g. communication mechanisms, decision making, administrative tasks, shared
goals and vision), meeting formats (e.g. types of meeting, frequency, or content) or
additional working groups (26). If the program specifies a clear organisational structure
which includes the elements mentioned above, the indicator can be marked as fulfilled.
5.2 Is a detailed implementation plan available which specifies how the program activities are to
be implemented?
A detailed manual is required which gives instructions and support on how to conduct the
components of the program. Such a manual or guide should contain an overview of the
processes and contents of the whole program. It does not only include a comprehensive
description of the program and its components, but also a detailed schedule list of required
materials and information about difficulties and how to overcome these. This should
support a high quality implementation phase. Also it ensures that the program is
implemented in a way, which is in line with the planning phase.
5.3 Is a detailed description of the costs/budget available which provides information on the
financial resources needed to carry out the program?
A detailed budget calculation and a funding plan is an important part of the program
structure and management. It is desirable that programs also present information on how to
obtain the resources through sponsoring or fundraising. When additional resources are
required, it is useful to describe these in detail, including time for the program realisation as
well as personnel and material resources.
5.4 Does the program provide training for those who implement the program?
If specific competencies are required to implement the program, a training program should
be available to prepare the people involved to deliver it. When no training program is
available, this could be seen as a lack of structural quality, which may lead to difficulties in
the implementation phase. The program should also provide information about the
frequency of trainings and the costs as well as the required attendance.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
20
6.1 Does the program provide easy access to relevant materials needed for program
implementation (e.g. through a website)?
Program dissemination is an important aspect of program impact. To ensure dissemination
(or ‘scaling up’) providers should ensure that all materials are easily accessible (online).
This could be done by providing copies of training manuals, facilitator guides, meeting
guides, planning tools, evaluation field guides and instruments through a website. When
program materials are easy accessible, it is more likely that the program is further
disseminated.
6.2 Does the program establish partnerships with local, regional, and national organisations in
terms of its dissemination?
Partnerships with a regional or national organisation with an infrastructure for
dissemination or partnering with local organisations can facilitate the dissemination.
Therefore it is recommended to establish partnerships with local, regional and national
organisations right from the beginning. A possibility to increase ownership and
responsibility could be to invite those organisations to be a part of the steering group of the
community based program.
3. Quality of process
7.1 Are the roles and responsibilities of organisations and persons who are involved in the
program clearly defined?
For a successful implementation of a program it is important to have clear responsibilities
and roles for each organisation and person involved in the program. A detailed project plan
of the whole program which gives guidance on what (content, e.g. milestones) needs to be
done and when (time table) by whom needs to be available. If there is no clear description of
the roles and responsibilities, ambiguities about who needs to do what can affect the success
rate of the implementation.
Is it apparent that the program tailors its activities to the specific context?
As described under indicator 4.5, community based programs should be implemented in
multiple settings. Each context within which the program intervenes has its own routines,
organisational practices, values and norms. For example, schools are educational
institutions which are characterized by their educational obligation, typical organisational,
time and personnel structures (e.g. lessons, classrooms, academic trained staff, a single staff
room) and procedures (e.g. teaching, grading). These features are different from those in
other settings (e.g. community health and youth services). Therefore each context needs its
own implementation strategy. The more the program fits its activities to the context, the
better it can be successfully implemented (10). Hence, the program needs to make use of
specific strategies to tailor its activities to the specific context.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
21
7.2 Are the marketing messages of the program delivered in a positive and non-stigmatizing
way?
Media play a significant role in the lives of children and adolescents. Since they are
susceptible for media messages, each communication message should be provided in a way
that avoids stigmatization (5). This can be achieved by using positive messages which
encourage health promotion (rather than moralizing messages related to problems and
risks).
7.3 Are concrete strategies applied to reach and keep disadvantaged children and adolescents
involved?
It is important for community based programs to reach and keep disadvantaged children
and adolescents involved by using an inclusive approach; participation barriers due to e.g.
costs, access, culture, education and language, should be avoided or reduced (5). Multiple
setting and multiple strategy oriented community programs are generally seen as an
equitable response to health related problems (27). But community based programs need
tailored strategies to tap their full potential. Some strategies include no participation fees,
participation incentives, the use of a central and accessible location, transport assistance
from and to the location, and bilingual sessions (28).
7.4 Does the program provide a monitoring tool which allows an ongoing assessment of the
implementation?
Process monitoring should be part of the quality management of the program which is
performed in the implementation phase. Monitoring the implementation reveals whether
improvements and adaptations within the implementation phase are necessary or not.
Monitoring indicators include an assessment of readiness for implementation, satisfaction
with the planned implementation and how acceptable it is as a choice of program. A well
designed community based program provides its users with a monitoring tool with which
they can assess the implementation and take actions to adapt the implementation if needed.
7.6 Is ongoing support (e.g. counselling) provided to those delivering the program?
Those responsible for carrying out the program should receive support in discussing
problems and difficulties in implementing the program. The program provider should offer
continuous support in addition to the initial training. This can include a helpdesk via
telephone or email or periodic meetings, counselling or supervision by an experienced
professional. The program should also give information about the frequency of available
support and its costs.
7.7 Does the program method fit with the principles of health promotion?
A community based program requires methods and strategies, which are based on
democracy and participation through active involvement of different groups like children
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
22
and adolescents, families, schools, health care centres etc. These should be based on equity
through equal access. Programs that fit health promotion principles are focused on the
empowerment of children and adolescents and support them to increase their action
competence. The methods include group work, discussion circles, shared actions, games,
simulations etc. They are not focused on moralising. Appropriate methods contain positive
messages, which motivate and support a sense of responsibility and manageability of one’s
own health.
8.1 Does the program try to establish active links with other relevant initiatives in the same
community?
To enhance dissemination of community based programs it is important to build links with
already existing initiatives and programs that also target health in the same community.
Through active networking it is possible to create synergy effects and to avoid duplication
effort. Hence, the community based program needs to be linked with other programs and
initiatives and to be made clear how this link works. Also, it must be clear that this
connection is still active through e.g. regular exchange meetings.
8.2 Does the program include capacity building on an individual, organisational, and
environmental level?
To cultivate and sustain health gains over time it is necessary that community based
programs take actions on every dimension of community capacity. According to the NSW
capacity framework three dimension of capacity can be distinguished: 1) health
infrastructure or service development; 2) program maintenance and sustainability and, (3)
problem solving capability of organisations and communities. These can be addressed by the
following five strategies:
1. Organisational Development (e.g. policies and procedures, strategic directions,
organisational structures, or management support);
2. Workforce development (e.g. professional development opportunities, professional
support and supervision, or undergraduate or post graduate degrees);
3. Resource allocation (e.g. financial resources, human resources, administrative
support, or decision making tools and models);
4. Leadership (interpersonal skills, strategic visioning and thinking, visioning the
future, or organisational management); and
5. Partnerships (e.g. shared goals, relationships, or implementation) (ibid.).
4. Quality of results
9.1 Are the evaluation methods and procedures based on agreed scientific standards?
Through the information which is available about the program it must be apparent that the
applied methods and procedures are rooted in sound scientific theory and research. This
includes detailed information about the study design, the underlying research questions
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
23
which emerged from the planning of the program, the indicators and variables and the
methods which were applied for data collection and statistical analysis. The study design
and research methods should be able to verify whether the observed effects are caused by
the program.
9.2 Do the program results demonstrate positive health outcomes?
A successful and effective program has to demonstrate a positive change in health related
parameters. Many indicators show whether the program is being effective or not. Positive
effects on the first outcome level are measured by looking at community structures and
practices followed by knowledge about health or attitudes and behavioural intentions.
Outcomes on an intermediate level are, for example, change in health behaviour such as an
increase of hours spent doing vigorous physical activity per week or an increase of fruit and
vegetable consumption. Other intermediate outcomes bear on a healthy environment like
the improvement of playgrounds. More difficult is the evidence of health outcomes like a
decrease in the number of overweight children and adolescents because this can not be
observed immediately. To meet the requirements of this indicator, the program should show
a positive outcome on the first and intermediate outcome level.
9.3 Is there evidence that the positive health outcomes sustain over a period of at least 6
months?
High quality programs must demonstrate that the positive effects persist after a time period
of at least six months. The longer a positive impact is evident, the more likely the child or
adolescent will continue this positive behaviour. In order to examine long term effects
follow up studies are required. Programs demonstrating that at least one positive effect can
be sustained over at least six months, fully meet the demands of this indicator.
9.4 Is the overall effort (costs, time) in balance with the effectiveness of the program?
If the program yields positive effects, it does not mean the program has been efficient. Part
of a cost-effectiveness analysis is to set all expenditures alongside the outcomes.
Expenditures relating to the program include costs for staff, material, time etc. whereas the
outcome is measured in non-monetary effects which are indicated by the effectiveness
evaluation.
The program can be assessed as being efficient if a cost-effectiveness analysis is conducted,
the methods and procedures are clearly described and the results demonstrate a positive
balance. Ideally the provider of the program will have compared cost-effectiveness results
with the cost-effectiveness findings of other community based programs on health
promotion. This requires that the programs used for comparisons are characterised by
similar conditions and features.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
24
10.1 Does the program have a positive impact on capacity building on an individual,
organisational, and environmental level?
Direct health outcomes such as reduced BMI or increased physical activity are important but
not the only relevant results indicating program success. When applying a socio-ecological
approach, the development of sustainable structures and processes through capacity
building on an individual, organisational and environmental level (see 8.2), must be a
primary goal of community based programs. Therefore the program needs to have a
demonstrable impact on capacity building in relation to health. The NSW capacity
framework which encompasses three dimensions and five strategies can be used for
orientation (see 8.2).
10.2 Has the program been successfully adopted by other communities (communities that were
not part of the development and evaluation)?
The success of a community based program can be measured by how often it has been
adopted by other communities not being part of the development phase (see 6.1).
Evaluation results should indicate whether and how often the developed materials (e.g.
copies of training manuals, facilitator guides, meeting guides, planning tools, evaluation field
guides and instruments) were implemented by other communities.
10.3 Have the results of the process evaluation been used to improve the program?
The outcomes of the process evaluation make clear if the program has been implemented as
intended (fidelity) and accepted by the participants. It also becomes clear if there are
implementation barriers which need to be overcome in order to improve the program. When
the results of the process evaluation have been used to improve the program the demands of
this indicator are met. This could encompass a description of strategies to increase
implementation readiness and motivation as well as recommendations on how to improve
the use of available resources (e.g. time, personnel).
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
25
Appendix 1 Glossary of basic terms
Term
Description
Action competence
Action competence is the ability to act and bring about
positive change with regard to health (29).
Benchmark
Benchmark can be defined as the use of comparative data
from similar interventions, activities etc. to set the standard
of best practice and therefore measure success and specific
needs for improvement.
BMI
Body mass index (BMI) is the index of weight-for-height that
is commonly used in classifying what it means to be
overweight or obese. It uses weight in kilograms divided by
the square of the height in meters (kg/m2) (30).
Community
A community is a group of people who share common values
and norms and are influenced by the same political situation
and environmental circumstances and whose members live
in a geographically defined area.
Coping
Coping refers to the person's cognitive and behavioural
efforts to manage (reduce, minimize, master, or tolerate) the
internal and external demands of the person-environment
transaction that is taxing or exceeding the person's
resources (31).
Cost-effectiveness
Cost-effectiveness indicates the relation between financial
expenditures on the one side and the outcome (impact) of
an intervention on the other. With a positive cost
effectiveness the outcomes exceed the financial
expenditures of the intervention.
Effectiveness
Adequacy of an intervention in terms of its intended effect
(32).
Efficiency
The relation between a result and the means employed to
achieve it (32).
Empowerment
In health promotion, empowerment is a process through
which people gain control over decisions and actions
affecting their health (33).
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
26
Evidence-based health
promotion
The use of information derived from formal research and
systematic investigation to identify causes and contributing
factors to health needs and the most effective health
promotion actions to address these in given contexts and
populations (34).
Health determinants
Determinants of health encompass a variety of factors which
influence the health status of individuals, groups or the
whole population. This includes the social and economic
environment, the physical environment as well as the
person’s individual characteristics and behaviours (33).
Health program
Health programs can be defined as actions which are
founded in health sciences and aim at systematically and
sustainably changing individual behaviour and/ or the
surrounding conditions. Two fundamental orientations can
be distinguished: health interventions which aim at the
promotion of health/ well-being (health promotion) and
interventions which are focused on the avoidance of
diseases (prevention) (1).
Health promotion
Health promotion is the process of enabling people to
increase control over, and improve their health (33).
Health risk
Social, economic or biological status, behaviours or
environments which are associated with or cause increased
susceptibility to a specific disease, ill health, or injury (33).
Implementation
Implementation refers to how well a proposed program or
intervention is put into practice (35). Different components
of implementation are implementation fidelity, frequency
and duration of the intervention, quality of delivery,
responsiveness of participants (36).
Intervention fidelity
Intervention fidelity or adherence refers to the question, if
and to which extent the whole intervention or intervention
components is/were delivered in the community setting as
planned (22). It reflects the degree of consistency between
the planning and the implementation.
Monitoring
Continuous or periodic systematic data collection used for
checking processes and results (37).
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
27
Overweight
Although there is no consensus on how to define what is
overweight in childhood and adolescence it is widely
common to use a gender and age specific percentile (value
below which a certain percent of observations fall). For
Europe ‘overweight’ is classified as at or above the 85th
percentile and obesity as at or above the 95th percentile of
BMI (38).
Participation
Participation happens at different levels and can range from
a sense of “taking part in” to “having part or share in
something”. The latter implies the sharing of power in the
decision making processes in relation to con textual matters
(like school). This understanding reflects a sense of self-
determination, ownership and empowerment in relation to
learning about health (33,39).
Prevention
Whereas health promotion is oriented at the strengthening
of health through the improvement of resources and
capabilities, prevention is oriented on the conservation of
health through avoidance of risks and diseases. Depending
on the time it can be differentiated between primary
prevention (avoidance of the genesis of a diseases),
secondary prevention (early diagnosis of diseases) and
tertiary prevention (avoidance of an elapse).
Prevalence
Number of cases of a defined population group at a certain
time (32).
Quality indicator
A specially selected measure that may indicate a good or
poor quality (40)
Secondary disorders
Secondary disorders are diseases, which were caused by an
earlier disease or event. Type II diabetes, high blood
pressure, heart attacks for example are secondary diseases
which can be caused by obesity.
Self-efficacy
Self-efficacy refers to beliefs that individuals hold about
their capability to carry out actions in a way that will
influence the events that affect their lives (34).
Self-esteem
Self esteem refers to the extent, to which a person values,
prizes or likes herself or himself. It describes a favourable or
unfavourable attitude towards the self (41).
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
28
Socioeconomic status (SES)
‘Socioeconomic status’ describes an individual or family’s
economic and social position in relation to others which is
usually determined by a family’s income.
Stress
Stress is produced through the relationship between a
person and their environment. More specifically an
environment that is felt to be taxing or exceeding their
resources and therefore as endangering wellbeing (31].
Sustainability
Intended impacts of a project which last longer than the
project itself (32).
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
29
References
1. Michaelsen-Gärtner, B., Witteriede, H. & Paulus, P. (2009). Gesundheitserziehung in Schulen.
[Health Education in Schools]. In B. Wulfhorst & Hurrelmann, K. (Ed.). Hand-buch
Gesundheitserziehung [Handbook of Health Education] (160-170). Bern, Hans Huber.
2. Donabedian, A. (2003). An Introduction to Quality Assurance in Health Care. New York,
Oxford University Press.
3. Ruckstuhl, B., Kolip, P. & Gutzweiler, F. (2001). Qualitätsparameter in der Prävention
[quality parameter in prevention]. In BZgA (Ed.). Qualitätsmanagement in
Gesundheitsförderung und Prävention [Quality management in health promotion and
prevention] (38-50). Köln, BZgA.
4. Trojan, A. (2001). Qualitätsentwicklung in der Gesundheitsförderung [Quality development
in health pro- motion]. In BZgA (Ed.). Qualitätsmanagement in Gesundheitsförderung und
Prävention [Quality management in health promotion and prevention] (51-72). Köln, BZgA.
5. King, L., Gill, T., Allender, S. & Swinburn, B. (2010). Best practice principles for community-
based obesity prevention: development, content and application. Obesity Reviews, 12, 329-
338.
6. Swinburn, B., Egger, G., Raza, F. (1999). Dissecting obesogenic environments: the
development and application of a framework for identifying and prioritizing environmental
interventions for obesity. Preventive Medicine, 29, 563-570.
7. Simmons, A., Mavoa, H. M., Bell, A. C. et al. (2009). Creating community action plans for
obesity prevention using the ANGELO (Analysis Grid for Elements Linked to Obesity)
Framework. Health Promotion International, 24, 311-324.
8. Bryant, C. A., Courtney, A. H., McDermott, R. J. et al. (2010). Promoting physical activity
among youth through community-based prevention marketing. Journal of School Health, 80,
214-224.
9. DeMattia, L. M., Denney, S.L. (2008). Childhood Obesity Prevention: Successful Community-
Based Efforts. Annals of the American Academy of Political and Social Science, 615, 83-99.
10. Bopp, M., Fallon, E. (2008). Community-based interventions to promote increased physical
activity: A primer. Applied Health Economics and Health Policy, 6, 173-187.
11. Mayer, K. (2009). Childhood Obesity Prevention Focusing on the Community Food
Environment. Family & Community Health, 32, 257-270.
12. Boureaudhuij de, I., van Cauwenberghe, E., Spittaels, H et al. (2011). School-based
interventions promoting both physical activity and healthy eating in Europe: a systematic
review within the HOPE project. Obesity Review, 12, 205-216.
13. Booth, S.L., Mayer, J, Sallis, J.F. et al. (2001). Environmental and Societal Factors Affect Food
Choice and Physical Activity: Rational, Influences, and Leverage Points. Nutrition Reviews,
59, S21-S36.
14. Bronfenbrenner, U. (1989). Ecological Systems Theorie. Annals of Child Development, 6,
187-249.
15. Birch, L.L., Davison, K.K. (2001). Family environmental factors influencing the developing
behavioral controls of food intake and childhood overweight. Pediatric clinics of North
America, 48, 893-907.
16. Rhee, K. (2008). Childhood Overweight and the Relationship between Parent Behaviors,
Parenting Style, and Family Functioning. The ANNALS of the American Academy of Political
and Social Science, 615, 11-37.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
30
17. Brug, J., Oenema, A., Ferreira, I. (2005). Theory, evidence and Intervention Mapping to
improve behavior nutrition and physical activity interventions. International Journal of
Behavioral Nutrition and Physical Activity, 2:2.
18. O´Dea, J., Caputi, P. (2001). Association between socioeconomic status, weight, age and
gender, and the body image and weight control practices of 6- to 19-year-old children and
adolescents. Health Education Research, 16, 521-532.
19. Stamatakis, E., Wardle, J., Cole, T.J. (2009). Childhood obesity and overweight prevalence
trends in England: evidence for growing socioeconomic disparities. International Journal of
Obesity, 34:41.
20. Eisenmann, J., Alaimo, K., Pfeiffer, K. et al. (2011). Project FIT: Rationale, design and baseline
characteristics of a school- and community-based intervention to address physical activity
and healthy eating among low-income elementary school children. BMC Public Health,
11:607.
21. Samuels, S. E., Craypo, L., Boyle, M. et al. (2010). The California Endowment’s Healthy Eating,
Active Communities Program: A Midpoint Review. American Journal of Public Health, 100,
2114-2123.
22. Silva-Sanigorski, A. de, Prosser, L., Carpenter, L. et al. (2010a). Evaluation of the childhood
obesity prevention program Kids - 'Go for your life'. BMC Public Health, 10.
23. Silva-Sanigorski, A. M. de, Bell, A. C., Kremer, P. et al. (2010b). Reducing obesity in early
childhood: results from Romp & Chomp, an Australian community-wide intervention
program. American Journal of Clinical Nutrition, 91, 831-840.
24. Silva-Sanigorski de, A. M., Bolton, K., Haby, M. et al. (2010c). Scaling up community-based
obesity prevention in Australia: background and evaluation design of the Health Promoting
Communities: Being Active Eating Well initiative. BMC Public Health, 10:65.
25. Ader, M., Berensson, K., Carlsson, P., Granath, M. & Urwitz, V. (2001). Quality indicators for
health promo- tion programmes. Health Promotion International, 16, 187-195.
26. Centers for Disease Control and Prevention (2003). Planned Approach to Community Health.
Guide for the Local Coordinator. Atlanta: CDC.
27. Silva-Sanigorski de, A., Economos, C. (2010). Evidence of multi-setting approaches for
obesity prevention: translation to best practice. In: E. Waters, B.A. Swinburn, J.C. Seidel, R.
Uauy (Ed.). Preventing Childhood Obesity. Evidence, Policy and Practice (p. 57-63). Oxford:
Blackwell Publishing.
28. Beaulac, J., Olavarria, M. & Kristjansson, E. (2010). A community-based hip-hop dance
program for youth in a disadvantaged community in Ottawa: Implementation findings.
Health Promot Practice, 11, 61S - 69S.
29. Jensen, B. B. (1997). A case of two paradigms within health education. Health Education
Research, 12, 419-428.
30. WHO (2006). Obesity and Overweight. Fact sheet N° 311. Retrieved from the World Wide
Web on August 11th, 2009 from http://www.who.int/mediacentre/factsheets/fs311/en/
index.html.
31. Folkman, S., Lazarus, R.S., Gruen, R.J. & DeLongis, A. (1986). Appraisal, Coping, Health
Status, and Psychological Symptoms. Journal of Personality and Social Psychology, 50, 571-
579.
32. Quint-Essenz (2008). Glossary. Retrieved from the World Wide Web on August 11th, 2009
from http://www.quint-essenz.ch/en/resources.
33. WHO (1998). Health promotion glossary. Geneva, WHO.
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
31
34. Smith, B.J., Tang, K.C. & Nutbeam, D. (2006). WHO Health Promotion Glossary: new terms.
Health Promotion International, 21(4), 340-345.
35. Durlak, J. A. (1998). Why Program Implementation is Important. Journal of Prevention &
Intervention in the Community, 17, 5-18.
36. Barry, M.M., Domitrovich, C., Lara, M.A. (2005). The implementation of mental health
promotion programmes. Promotion & Education, S2, 30-36.
37. Shaya, F. T., Flores, D., Gbarayor, C. M. & Wang, J. (2008). School-Based Obesity
Interventions: A Literature Review. Journal of School Health, 78, 189-196.
38. Summerbell, C. D., Waters, E., Edmunds, L. D., Kelly, S., Brown, T. & Campbell, K. J. (2005).
Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews,
20, 1-73.
39. BZgA (2006). Quality criteria for programmes to prevent and treat overweight and obesity
in children and adolescents. Cologne, BZgA.
40. Ader, M., Berensson, K., Carlsson, P., Granath, M. & Urwitz, V. (2001). Quality indicators for
health promo- tion programmes. Health Promotion International, 16, 187-195.
41. Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ, Princeton
University Press.
Useful web resources
1. Quality instruments for health interventions (selection)
Criteria for good practice in health promotion for social disadvantaged people (only
available in German)
http://www.bzga.de
European quality instrument for health promotion (EQUIHP)
http://subsites.nigz.nl/systeem3/site2
Preffi
http://www.preffi.nl
Quality in prevention (only available in German)
zhttp://www.uke.de/extern/qip
Quality criteria for programs to prevent and treat children and adolescents who are
overweight and obese
http://www.bzga.de
Quint Essenz
http://www.quint-essenz.ch
2. Databases relating to healthy eating and activity (selection)
Platform for eating and activity (only available in German)
http://www.ernaehrung-und-bewegung.de
Platform health promotion for socially disadvantaged (only in German)
http://www.gesundheitliche-chancengleichheit.de
I-database (only available in Dutch)
http://www.loketgezondleven.nl/i-database
euPrevent | EMR, Postbus 661, 6200 AR Maastricht (NL)
www.euPrevent.eu
32
Healthy school database Austria (only available in German)
http://gs.bmgfj.gv.at
3. Other relevant web resources
Schools for Health in Europe (SHE) network
http://www.schoolsforhealth.eu
HEPS supports school health policy
http://www.hepseurope.eu
