Content uploaded by Dan A. Olweus
Author content
All content in this area was uploaded by Dan A. Olweus on Jun 16, 2017
Content may be subject to copyright.
377
27
e Olweus Bullying Prevention Program
Implementation and Evaluation over Two Decades
DAN OLWUS AND SUSAN P. LIMBER
In 1983, a er three adolescents in Norway committed suicide, most likely as a consequence
of severe bullying by peers, the Norwegian Ministry of Education initiated a nationwide cam-
paign against bullying in schools. What has later become known as the Olweus Bullying Preven-
tion Program (OBPP) was developed and initially evaluated within this context.
In this chapter, we will describe the conceptua l basis for the program and summarize ndings
from the rst evaluation. We then will describe in some detail the components of the program
as typically used in the United States and summarize subsequent evaluations of the program,
primarily in Norway and the United States.
Goals and Basic Principles of the OBPP
e main goals of the OBPP are to reduce existing bullying problems among students at school,
prevent the development of new bullying problems, and more generally, achieve better peer rela-
tions at school (Olweus, 1993a; Olweus, Limber, & Mihalic, 1999; Olweus et al., 2007). ese
goals are met through a restructuring of the child’s social environment at school. e restruc-
turing is intended to reduce both opportunities and rewards for engaging in bullying behavior
and to build a sense of community among students and adults within the school environment.
Positive, pro-social behaviors are encouraged and rewarded (Olweus, 1993a, 2001a; Olweus et
al., 2007).
e OBPP is based on four key principles: Adults at school should: (a) show warmth and
positive interest and be involved in the students’ lives; (b) set rm limits to unacceptable behav-
ior; (c) consistently use nonphysical, nonhostile negative consequences when rules are broken;
and (d) function as authorities and positive role models (Olweus, 1993a, 2001a; Olweus et al.,
2007). ese principles are derived chie y from research on the development and modi cation
of the implicated problem behaviors, in particular aggressive behavior (Baumrind, 1967; Loeber
& Stouthamer-Loeber, 1986; Olweus, 1973, 1978, 1979, 1980). ese listed principles have been
translated into a number of speci c measures to be used at the school, classroom, individual,
and, in some contexts, the community level. ese components will be described in some detail
below.
In The Handbook of Bullying in Schools: An International MAY BE CITED
Perspective (2010, pp. 377-401). Edited by Shane R.
Jimerson, Susan M. Swearer, and Dorothy L. Espelage.
New York: Routledge.
378 • Dan Olweus and Susan P. Limber
Initial Evaluation of the OBPP
e OBPP was rst implemented and evaluated in the First Bergen Project against Bullying, a
longitudinal study that followed approximately 2,500 school children over a period of two-and-
a-half years, from 1983 to 1985. Because the project was part of a nationwide campaign against
bullying, it was not possible to conduct an experimental study with schools or classes randomly
assigned to treatment and control conditions. Instead, an extended selection cohorts design (see
below) was utilized in which same-aged students from the same schools were compared across
three points in time (Olweus, 1991, 1993a. 1997, 2005). At Time 1, participants belonged to 112
classes in grades 5–8 from 28 elementary and 14 junior high schools in Bergen, Norway. e
number of students participating in the 1983–84 evaluation (grades 6 through 8) was approxi-
mately 1,750 at each time point. e corresponding 1983–85 evaluation (grades 7 and 8) was
based on approximately 1,210 students.
Results from the evaluation revealed marked (and statistically signi cant) reductions in self-
report of bully/victim problems (based on an early version of the Olweus Bully/Victim Ques-
tionnaire, 1986). In the 1983–84 evaluation, the relative reduction for being bullied was 62.0%
(from 10.0% to 3.8%) and 33.0% for bullying other students (from 7.6% to 5.1%). Corresponding
results for the 1983–85 evaluation, involving the grade 7 and 8 students, were 64.0% (from 10.0%
to 3.6%) and 52.6 % (from 7.6% to 3.6%) (Olweus, 1991, 1993a, 1997; Olweus & Alsaker, 1991).
Similar results were obtained for two aggregated peer rating variables (class-aggregated peer
estimates of “number of students being bullied in the class” and “number of students in the class
bullying other students”) and teacher ratings of bully/victim problems at the classroom level.
With regard to the teacher data the e ects were somewhat weaker. A marked “dosage-response”
relationship (r = .51, n = 80) was observed, such that those classes in which essential components
of the program (classroom rules against bullying, use of role playing, and classroom meetings)
had been implemented experienced greater reductions in bullying problems (Olweus & Alsaker,
1991; Olweus & Kallesbad, in press). Finally, the study also documented signi cant reductions
in self-reports of general antisocial behavior, including vandalism, the , and truancy, and
improvements in aspects of the “social climate” of the class: Improvements were seen in stu-
dents’ self-reports of satisfaction with school life, improved order and discipline, more positive
social relationships, and a more positive attitude toward school work and the school in general.
Detailed analyses of the quality of the data and the possibility of alternative interpretations
of the ndings led to the following general conclusions: It is very di cult to explain the results
obtained as a consequence of (a) underreporting by the students; (b) gradual changes in the stu-
dents’ attitudes to bully/victim problems; (c) repeated measurement; or (d) concomitant changes
in other factors, including general time trends (Olweus, 1991).
Factors A ecting Program Implementation
Research and experience indicate that implementation of the OBPP can vary substantially
among teachers and schools (Kallestad & Olweus, 2003; Limber, 2006; Olweus, 2004). System-
atic examinations of factors that a ect implementation of prevention programs are relatively
scarce but critical. As Biglan (1995) noted, “the adoption of an e ective practice is itself a behav-
ior in need of scienti c research” (p. 51).
In an attempt to understand teacher- and school-level factors that a ect implementation of
the OBPP, Kallestad and Olweus (2003) analyzed responses of 89 teachers from 37 schools and
assessed program implementation at two points in time in the First Bergen Project against Bul-
lying. Although this multilevel study was not published until in 2003, the implications of the
e Olweus Bullying Prevention Program • 379
analyses were known and incorporated into the OBPP and its model of implementation long
before publication of the paper. A detailed discussion of our ndings and conclusions is beyond
the scope of this chapter but can be found elsewhere (Kallestad & Olweus, 2003; Olweus, 2004).
Here only a few general points will be emphasized.
e results from this study clearly indicated that the teachers were key agents of change
with regard to adoption and implementation of the OBPP. Substantial amounts of variance in
implementation could be predicted on the basis of teacher-level variables such as “perceived
sta importance” (teacher e cacy), “having read (more of) the program materials,” and “a ec-
tive involvement” (empathic identi cation with victims of bullying). Also school-level variables
such as “openness in communication among sta ” (positive attitude to change) and the “school’s
attention to bullying problems” contributed substantially to prediction. is study shed light
on several factors of importance and contributed to a better understanding of the process of
program implementation. e results also suggested ways in which the program itself and its
implementation could be improved.
Cultural Adaptation
Although the basic principles of the OBPP and its core components have remained largely
unchanged, research and extensive experience from implementing the program in the eld have
naturally led to some adaptations of the program to di erent cultural contexts. e current
employment conditions of teachers in the United States, for example, are such that less time
can be devoted to sta discussion group meetings (below) than in Norway. As a consequence,
the Bullying Prevention Coordinating Committee has been assigned a somewhat di erent role
and greater responsibility for implementation of the program in the United States than in Nor-
way (Limber, 2004). e organization of the training of trainers (named instructors in Scan-
dinavia) also di ers somewhat in the two countries (Olweus et al., 2007; Olweus, 2001a, 2004).
Similarly, in line with much American tradition and recognizing the importance of commu-
nity involvement in prevention e orts more generally and outside the school context, recent
implementations of the program in the United States have encouraged schools also to include
community-level components.
Table 27.1 summarizes the components of the OBPP at each of the four levels of focus as typi-
cally used in the United States the school, the classroom, the individual, and the community.
Below, we will describe components at each of these levels of OBPP. Two recently published
manuals, a Teacher Guide (Olweus & Limber, 2007) and a Schoolwide Guide (Olweus et al., 2007),
provide much more detail about the program and its components for interested readers.
School-Level Components
Typically, the following eight program components are implemented schoolwide.
Bullying Prevention Coordinating Committee e Bullying Prevention Coordinating
Committee (BPCC) is a building-level committee that is responsible for ensuring that all
components of the OBPP are implemented in a school. e BPCC is a representative team from
the school and broader community, which typically is comprised of 8 to 15 members, including
a school administrator, a teacher from each grade level, a school counselor and/or school-based
mental health professional, a representative of the non-teaching sta (such as a bus driver,
custodian, or cafeteria worker), one or two parents, a representative from the community (such
as sta from an a er-school or youth organization or key representatives from the business or
380 • Dan Olweus and Susan P. Limber
faith community), and other school personnel who may bring particular expertise (such as a
nurse, school resource o cer, Title IX representative). Where appropriate, one or two student
representatives may also serve on the committee (typically middle, junior, or high school
grades). As an alternative, a separate student advisory committee may be formed to help ensure
meaningful participation by students in the planning and implementation of the program.
e responsibilities of the committee are to attend an intensive 2-day training by a certi ed
OBPP trainer; develop a plan to implement the OBPP at their school; communicate the plan to
sta , parents, and students; coordinate the program with other prevention/intervention e orts
at the school; obtain ongoing feedback from sta , parents, and students about the implementa-
tion of the OBPP and make adjustments to the school plan as needed; and represent the program
to the broader community (Olweus et al., 2007). e BPCC typically is chaired by an on-site
OBPP coordinator, who may be a school counselor, administrator, prevention specialist, or a
member of the non-teaching sta . e committee meets regularly throughout the life of the
program (at least monthly for the rst year of the program).
Training and Consultation In addition to the 2-day training provided to members of the
BPCC, a certi ed Olweus trainer provides at least 1 year of in-person or telephone consultation
to the school’s on-site coordinator to help ensure delity to the model and to problem-solve as
needed. Members of the BPCC (o en with assistance from the trainer) provide a full day of
training for all school sta prior to launching the program. Supplemental trainings that provide
more intensive attention to particular topics (e.g., cyber bullying, classroom meetings) also may
be held, as are yearly catch-up trainings for new sta .
Administration of the Olweus Bullying Questionnaire e Olweus Bullying Questionnaire
is a validated, self-report survey that assesses students’ experiences with and attitudes about
Table 27.1 Components of the OBPP
School-Level Components
• Establish a Bullying Prevention Coordinating Committee.
• Conduct committee and sta trainings.
• Administer the Olweus Bullying Questionnaire schoolwide.
• Hold sta discussion group meetings.
• Introduce the school rules against bullying.
• Review and re ne the school’s supervisory system.
• Hold a school kick-o event to launch the program.
• Involve parents.
Classroom-Level Components
• Post and enforce schoolwide rules against bullying.
• Hold regular class meetings.
• Hold meetings with students’ parents.
Individual-Level Components
• Supervise students’ activities.
• Ensure that all sta intervene on the spot when bullying occurs.
• Conduct serious talks with students involved in bullying.
• Conduct serious talks with parents of involved students
• Develop individual intervention plans for involved students.
Community-Level Components
• Involve community members on the Bullying Prevention Coordinating Committee
• Develop partnerships with community members to support your school’s program.
• Help to spread anti-bullying messages and principles of best practice in the community.
e Olweus Bullying Prevention Program • 381
bullying (Olweus, 2007; Solberg & Olweus, 2003). e questionnaire is administered (usually
anonymously) to students in grades 3–12 prior to the implementation of the program and at
regular (typically yearly) intervals therea er. e questionnaire provides a detailed de nition of
bullying and, among other questions, asks students to disclose the frequency with which they
have experienced and participated in various forms of bullying in the past couple of months. e
current scannable questionnaire (for use in the United States, Olweus, 2007) has 40 standard
questions, including some new questions about students’ experiences with cyber bullying. A
standard report of ndings is produced for schools, which provides detailed information (in
tables, graphs, and narrative) about the ndings, frequently broken down by gender and grade.
Schools use results from the survey to help raise awareness among students, sta , and parents
about the problems of bullying; make speci c plans for the implementation of the OBPP; and
assess change over time (for girls and boys and across grade levels).
Sta Discussion Groups Discussion groups of teachers and other school sta meet on a regular
basis in order to learn about and have in-depth discussions about the OBPP and re ect on bul lying
and related prevention e orts at the school. ese groups typically consist of no more that 15
personnel and are led by a member of the BPCC. It is recommended that they meet at least once
per month for the rst year of the program, and somewhat less frequently a er that.
School Rules and Positive and Negative Consequences Schools are asked to adopt four speci c
rules about bullying.
1. We will not bully others.
2. We will try to help students who are bullied.
3. We will try to include students who are le out.
4 If we know that somebody is being bullied, we will tell an adult at school and an adult at
home.
e rules cover both direct and indirect forms of bullying, including social isolation and
intentional exclusion from the peer group (Olweus & Limber, 2007). ey are posted throughout
the school and discussed with students and parents. Teachers and other sta are taught how to
apply consistent positive and negative consequences to reinforce these rules.
Supervisory System e BPCC in each school reviews and re nes its supervisory system
with the goal of reducing bullying behavior. is review includes determining “hot spots”
for bullying based on results of the Olweus Bullying Questionnaire, developing strategies to
increase supervision in common “hot spots,” developing means of tracking and reporting
bullying, assessing the attitudes and skills of supervising adults, and evaluating the school’s
physical design to reduce bullying.
School Kick-O Event Each school launches its program with students with an individually-
designed event that is intended to increase awareness about bullying, introduce the OBPP
to students, and clarify the school’s rules and procedures relating to bullying. Schools are
encouraged to hold a kick-o each subsequent year to introduce the program to new students
and remind returning students about the school’s e orts to prevent bullying.
Parent Involvement Parents are viewed as important partners in preventing and intervening
in bully ing. ey are involved in t he OBPP in a number of ways, including ser ving on the school’s
coordinating committee, attending kick-o events and/or schoolwide parents meetings, and
382 • Dan Olweus and Susan P. Limber
receiving regular information about bullying and the OBPP through brochures, newsletters,
events, and online bulletin boards.
Classroom-Level Components
As illustrated in Table 27.1, there are three classroom-level components of the OBPP.
Post and Enforce School-Wide Rules Against Bullying Teachers discuss in detail with their
students (for example, in classroom meetings, below) the schoolwide rules about bullying to
make sure the students have a clear understanding of what they are and mean. e rules are
posted in every classroom for easy reference, and they are enforced using consistent positive and
negative consequences. is common set of rules sends a signal to students, parents and others
that the school has a uni ed and coordinated policy against bullying. ese rules should be an
independent part of the school’s discipline policy (Olweus & Limber, 2007).
Classroom Meetings Regular (weekly) class meetings in which teachers and students discuss
bullying and related issues are important components of the OBPP. Purposes of the class
meetings are to build class cohesion and community, discuss rules about bullying and positive
and negative consequences for following or not following the rules, help students understand
their roles in preventing and stopping bullying, and problem-solve strategies for addressing
bullying. As part of these meetings, students engage in role-playing, which is intended to help
build empathy and perspective-taking skills, generate possible solutions to bullying situations,
and practice positive actions to take when confronted with bullying.
Classroom Meetings with Parents Within the classroom setting, teachers are encouraged
to hold several classroom-level meetings with parents about the OBPP. e purposes of these
meetings are to help parents understand problems associated with bullying and ways that the
school is addressing bullying through the OBPP, and to solicit parental input and involvement
in the program (Olweus & Limber, 2007). Small, classroom-level meetings with parents also may
help to build rapport with the teacher and build connections among parents of students in the
class.
Finally, although not a core classroom-level component of the OBPP, schools are encouraged
to integrate bullying prevention messages and strategies throughout the curriculum.
Individual-Level Components
As illustrated in Table 27.1, there are several individual-level components of the OBPP. First,
sta are encouraged to increase supervision of students, particularly in known “hot spots” for
bullying, and particularly of students who are known or suspected to be involved in bullying.
Second, all sta are trained to intervene on-the-spot whenever they observe bullying. Speci c
procedures also have been developed to guide sta in how to react when a bully/victim problem
has been identi ed or is suspected in a classroom. ese procedures include serious talks with
the child who is suspected of being bullied and his or her parents and a er that, similar talks
with the child who is suspected of bullying, and his or her parents. A clear message is commu-
nicated that the bullying will be stopped and that the situation will be closely monitored (see
Olweus & Limber, 2007, pp. 87–106). In some schools, these meetings are conducted by counsel-
ors or administrators, but wherever possible, it is recommended that the meeting be led by the
children’s primary teacher or the sta member with the closest relationship with the student(s)
e Olweus Bullying Prevention Program • 383
involved. Finally, schools are encouraged to work with parents and mental health professionals
to develop individualized plans, where appropriate, to provide support and other help for stu-
dents who have been bullied and to correct the behavior of students who bully other students.
Community-Level Components
Recognizing that bullying does not stop at the doors of a school, committee members are
encouraged to involve one or more community members on their BPCC, look for ways that
community members can support the school’s bullying prevention program, and collaborate to
spread bullying prevention messages and strategies beyond the school into community settings
where children and youth gather.
Subsequent Outcome Studies of the OBPP
Subsequent to the initial evaluation of the OBPP in the First Bergen Project against Bullying, six
follow-up outcome evaluations of the OBPP have been conducted in Norway. e program has
also been evaluated in several diverse communities in the United States, including rural South
Caroli na (Li mber, Nation, Trac y, Melton, & Flerx, 2004; Melton et al., 1998), inner-city Phi ladel-
phia (Black & Jackson, 2007), suburban Chula Vista, California (Pagliocca, Limber, & Hashima,
2007), and Washington state (Bauer, Lozano, & Rivara, 2007).
In addition, several school-based anti-bullying programs inspired by the OBPP have been
implemented and evaluated in various countries including Belgium (Stevens, de Bourdeaud-
huij, & Van Oost, 2000; Stevens, Van Oost, & de Bourdeaudhuij, 2004), Canada (Pepler, Craig,
Ziegler, & Charach, 1994; Pepler, Craig, O’Connell, Atlas, & Charach, 2004), Gemany (Hanewin-
kel, 2004), and the United Kingdom (Whitney, Rivers, Smith, & Sharp, 1994; Eslea & Smith,
1998; Smith, Sharp, Eslea, & ompson, 2004). e results from these studies have been mixed,
with both positive and negative (null) results, and it is important to emphasize that these studies
cannot be seen as replications of the OBPP. e programs used in these interventions have devi-
ated considerably, but to di erent degrees, from the OBPP model in terms of program content,
implementation model, or actual implementation.
In this context, a study in the county of Rogaland, Norway, should be mentioned. In a short
book chapter, Roland (1989) has claimed that an intervention study parallel to the one in Bergen
was conducted in Rogaland with primarily negative results when outcome data were collected
a er 3 years, ending in 1986. In several respects, this account is grossly misleading. In summary,
it has repeatedly been shown (e.g., Olweus, 1999a, 2004) that the studies in Bergen and Rogaland
were two very di erent projects in terms of planning, data quality, times of measurement, and
contact with the schools, and accordingly, also in terms of expected results. We nd it neces-
sary to point this out since several authors in the eld still continue to erroneously present the
Rogaland project as an (unsuccessful) evaluation of the OBPP.
Methodological Comments
Design Issues Before describing the individual outcome studies, some methodological
comments are in order. With the exception of the New Bergen Project against Bullying (Olweus,
1999b, 2005), the South Carolina project (Melton et al., 1998), and the Washington state study
(Bauer et al., 2007), which used a traditional (non-randomized) control group design, and the
Philadelphia study (Black & Jackson, 2007), all evaluation studies have used the extended version
of the selection cohorts design (Olweus, 2005; Cook & Campbell, 1979; sometimes called age-
384 • Dan Olweus and Susan P. Limber
cohort design, Judd & Kenny, 1981). In this design, the data from two presumably equivalent
(grade or school) cohorts of students are compared at two (or more) time points. One cohort
provides data for Time 1 (before intervention) and the other data for Time 2, typically 1 year later
and a er approximately 8 months of intervention. In order to rule out explanations of the results
in terms of di erences in age or maturation, possible developmental changes are controlled by
comparing age-equivalent groups/cohorts at the various time points. In our studies, for example,
Grade 7 students at Time 2 (intervention group) are compared with same-aged Grade 7 students
from the same schools at Time 1 (control condition, before intervention). Similar comparisons
can be made for sets of grade cohorts such as grades 4 through 7 which cohorts have been the
main target groups for the OBPP.
A strength of the design is that the majority of the members in the various grade cohorts have
been recruited from the same, usually relatively stable populations and have been students in the
same schools for several years. Consequently, there are o en good grounds for assuming that a
cohort measured at Time 1 di ers only in minor ways from the adjacent cohort with which it is
compared at Time 2. Another advantage is that several of the cohorts serve both as a baseline
(Time 1) group in one set of comparisons and as an intervention (Time 2 or Time 3) group in
another. is fact will serve as partial protection against selection bias (see Olweus, 2005).
In any study designed to evaluate the possible e ects of an intervention program (or other
similar factor), it is mandatory that the researcher examines and ideally is able to rule out most
or all alternative explanations of the results in terms of possible confounding, “irrelevant” fac-
tors. is is true whether the study is experimental with randomization of subjects on condi-
tions or quasi-experimental (Cook & Campbell, 1979).
e logic of the extended selection cohorts design (a quasi-experimental design) and possible
threats to the “internal validity” of this design are discussed in considerable detail elsewhere
(Olweus, 2005). One particular threat to consider in this design concerns the possibility that
changes in the outcome variable at Time 2 (or Time 3) are a consequence of some irrelevant
factor concomitant to the intervention, implying that obtained results can be given a “history
interpretation.” is threat may be di cult to completely rule out in a single study if little is
known about results from studies with the same outcome variables without intervention. How-
ever, as regards the “being bullied” and “bullying other students” variables, which are regularly
used in evaluations of the OBPP, there are now a number of large-scale Norwegian popula-
tion studies without systematic intervention, which have shown very small or no changes for
comparable cohorts in successive years (Furre, Danielsen, Stiberg-Jamt, & Skaalvik, 2006). Such
ndings certainly support an interpretation of positive intervention results in terms of e ects of
the intervention rather than of some other concomitant factor, including time trends and media
attention.
Moreover, when positive results are repeatedly obtained in adequate replications, under
somewhat varying conditions and in di erent time periods, this of course makes a history inter-
pretation of the ndings much more unlikely. In addition, the structure of the data obtained in
the New National Initiative (described below) can shed a special light on the reasonableness of
a history interpretation of our ndings with the OBPP. is point will be discussed in a later
section of the chapter.
Statistical Analyses and E ect Sizes In all studies over which we have had some control, our
statistical analyses have taken the hierarchical or nested structure of the data into account, with
students nested within classrooms nested within schools. is is necessary to get correct (not
too small) estimates of standard errors and thereby adequate tests of the signi cance of the
intervention e ects. One way of doing this is to adjust the magnitude of the standard errors
e Olweus Bullying Prevention Program • 385
(upwards) in a way that corresponds to the degree of dependence among the students (data)
within the higher unit (by multiplication of the “ordinary” standard errors by the “design
e ect factor”; see Kish, 1987, p. 203, for example). In the more recent projects, we have used so
called mixed models with the SPSS program or multilevel models with the HLM and/or Mlwin
so ware packages. If the ordinary General Linear Model with standard t- or F-tests is used
on nested data—a practice that is still quite common—the standard errors will be too small,
resulting in too many signi cant ndings. is problem has been pointed out in a number of
publications over the past 10–15 years (e.g., Bryk & Raudenbush, 1992; Murray, 1998; Zucker,
1990) but many “interventionists” do not seem to be aware of this problem. As a consequence,
many studies of intervention e ects are incorrectly analyzed and may be considerably biased in
a positive direction.
In several of our Norwegian evaluation studies, we have not provided much statistical detail
with speci ed t- or F-tests and associated p-values. Although the standard alpha levels of .05, .01
may be considered a kind of benchmark against which the results obtained can be compared,
such comparisons become less meaningful when the sizes of the samples involved are large or
very large, which is the case with most of our studies. Under such conditions, even very small
di erences or changes become statistically signi cant. However, in various overview papers of
our studies we have o en noted that all main conclusions about the e ects of the intervention
program are based on results that are statistically signi cant or highly signi cant (e.g., Olweus,
1991, 2005).
Against this background, we have typically found it more meaningful to focus on some form
of measure of the size of the intervention e ects. A common e ect size measure is Cohen’s d
(Cohen, 1988), the standardized mean di erence, which is a useful measure for many purposes
and situations. However, for universal intervention programs, which aim to target all students
in a unit such as a school, a measure like Cohen’s d (or a point biserial correlation or some
variant of d) will be misleading, giving estimates that are too small, o en much too small. e
reason is that in most bullying intervention studies, a large proportion of the students, perhaps
60%–80%, are not directly involved in the problems (i.e., have a score of zero on relevant out-
come variables such as being bullied or bullying other students) and thus cannot change to a
lower score. Normally, the e ect of an intervention is measured as the di erence in proportions
(or some similar measure) of “problem” students who have been exposed or not exposed to the
intervention. With universal programs, however, the inclusion of a large group of “non-prob-
lem”/not-involved students (who cannot “improve”) in the analyses will considerably “dilute”
the e ects of the program when e ect is indexed by measures such as Cohen’s d.
As a consequence, to estimate e ects of the intervention, we have typically used a measure of
Relative (percentage) Change which is calculated as the di erence in percentages between the
Control condition (the Time 1 measure in a selection cohorts design) and the Intervention con-
dition 1 year later (the Time 2 measure for students in the same grades and schools as at Time 1
in a selection cohorts design) divided by the Control condition value. To express the value as a
percentage, the di erence in the numerator is multiplied by 100. To illustrate, if the percentage
of bullied students in Grade 6 is 20% at Time 1 and there are 15% bullied students among the
(former Grade 5, “now”) Grade 6 students at Time 2 (a er 8 months of intervention), the Rela-
tive Change score will be: (20-15)*100/20 = 25% reduction. However, if there are more bullied
students at Time 2, this increase will come out as a negative Relative Change value. e relative
change measure can also be used (with a slight modi cation) in a standard pre-post design with
a control group and an intervention group measured at two time points. It is worth noting that
the relative change measure is closely related to the measure of relative risk ratio (and Odds
ratio when the relative risks are low) used in many epidemiological studies (Fleiss, 1994). ese
386 • Dan Olweus and Susan P. Limber
measures are not perfect with universal programs, but they give a better and more meaningful
impression of what change has actually been achieved. It is o en also useful to report or look at
the absolute percentage change. In the example above, there would be a 20%–15% = 5% absolute
reduction gure which can be used to estimate the number of students (in the school, school
district, or larger unit) who have changed status from being bullied to not being bullied.
Outcome Variables, Gender Di erences, and Grade Focus For ease of understanding and
convenience, most of our reports have used dichotomized versions of being bullied and bullying
other students (“2 or 3 times a month” or more o en) as outcome variables. ere is a good
deal of information about the psychometric properties of these variables (Solberg & Olweus,
2003; Olweus, 2007; Kyriakides, Kaloyirou, & Lindsay, 2006), and they have been found to
function well for both prevalence estimation and the measurement of change (Olweus, this
volu me). Howe ver, as is wel l known, powe r is we akene d when graded or cont inuou s va riabl es a re
dichotomized. Accordingly, when the statistical analyses have been conducted on the graded,
non-dichotomized versions of these variables, the results in terms of statistical signi cance have
typically been even stronger. e same is true when indexes of several related items such as the
mean of all forms of being bullied (or bullying other students) have been used.
In addition, it should be mentioned that in a number of more detailed analyses we have usu-
ally also found changes in related variables re ecting increased intervention e orts on the part
of the schools as viewed by the students. ese data strongly support the interpretation that the
positive results obtained are a consequence of the intervention. To illustrate, students at Time 2
(intervention) have typically reported more active intervention in bullying situations from both
teachers and peers in comparison with reports from Time 1 students. Also, at Time 2 more stu-
dents have responded that the homeroom/main classroom teacher had done “much” or “a good
deal” to counter bullying in the classroom in the past few months. Due to space limitations, we
will not focus on such results in this chapter.
In most of our evaluation studies in Norway, the registered relative changes in bully/victim
problems have been relatively similar among boys and girls. Accordingly, not to overwhelm the
reader with data, most of our results will be presented for boys and girls combined.
As mentioned, the main target groups of the OBPP have been students in grades 4 through 7
(which represent the elementary grades in Norway; modal ages 10 though 13), and most of the
evaluation results in this chapter will concern students in these grades. e program has also
been used with students in higher grades, although results for the typical junior high school
grades (8 through 10) have been less consistent than for the lower grades. In a later section, we
will also comment brie y on results for these grades.
e Nature of the Student Populations Studied A brief comment on the nature of the
popu lations of students stud ied in Norway is also in order. ere seems to be a common view, in
p a r t i c u l a r a m o n g U . S . re s e a r c h e rs / r e a d e r s , t h a t No r w e g i a n / S c a n d i n a v i a n s t u d e nt s a r e m or e w e l l -
adjusted and less aggressive than their counterparts in American schools and, by implication,
presumably easier to change with an anti-bullying program such as the OBPP. Such a view may
not be unreasonable if comparison is made with American schools in very disadvantaged, inner
ci ty ar ea s. H owe ver, i f co mpa ris on is m ad e be tw ee n na ti ona lly re pre se nta ti ve s am ple s of s tu dent s
in the United States and Norway, there is little support for such a view, with several empirical
studies actually providing evidence to the contrary. For example, the international Program
for International Student Assessment study from 2002 (PISA; conducted at 3-year intervals),
which contains data from large, nationally representative samples of 15-year-old students from
29 OECD countries including the United States, contained a set of questions about noise and
e Olweus Bullying Prevention Program • 387
disruptive behavior in the classroom. Norway received a hardly attering top ranking (worst of
all countries) while U.S. students occupied rank number 24, close to the bottom-ranked (best)
Japanese students. In the PISA study of 2000, the results were quite similar indicating that the
measured characteristics are fairly stable over time (Kjærnsli, Lie, Olsen, Roe, & Turmo, 2004,
2006).
In the International Health and Behaviour of School Children study (HBSC; Craig & Harel,
2004, with data from 2001/2002), which contains the two global questions from the Olweus Bul-
lying Questionnaire, the levels of problems were fairly similar for United States and Norwegian
students in grades 6 and 8. e same was t rue of “ having been involved i n physic al ghting three
or more times in the previous 12 months.” is study, which is carried out every 4 years, uses
large nationally representative samples of students from selected grades. In the last two assess-
ments (2001/2002 and 2005/2006), more than 30 countries participated in the study. All of these
data clearly suggest that the common view of vast behavioral di erences between Norwegian
and “ordinary” U.S. students is greatly exaggerated and needs to be reconsidered.
In addition, it should be mentioned that the schools in Oslo, the capital of Norway, have an
average of about 25% of students with an immigrant background, a considerable proportion of
which come from non-Western countries. e OBPP has been implemented in some 40 Oslo
schools with equally good results as in the rest of the country (see below).
Norwegian Outcome Studies with the OBPP 1997–2000
e New Bergen Project against Bullying In this longitudinal project, Olweus (1999b, 2005)
and members of the Olweus Bullying Prevention Research Group assessed approximately 2,400
students in grades 5, 6, and 7 from 11 intervention and 11 comparison schools. To make results
comparable across studies in this chapter, the data for the two sets of schools were analyzed as
two selection cohorts designs for two time points.
e schools were not randomized by conditions but had approximately the same levels of
bully/victim problems at the start of the study (Time 1, May, 1997). One year later, a relative
reduction of 23.6% in being bullied (from 12.7% to 9.7%) was registered for the h- to sev-
enth-grade students in the intervention schools whereas comparable students in the comparison
schools evidenced a small increase by 4.7% (from 10.6% to 11.1%). For bullying other students,
the negative results for the comparison schools were more marked. ese schools had a relative
increase of 36.6% at Time 2 (from 4.1% to 5.6%) whereas the intervention schools showed a rela-
tive reduction of 21.4% (from 5.6% to 4.4%).
If one just looks at the results for the intervention schools, the results are less marked than in
the First Bergen Project against Bullying. It is not unreasonable, however, to combine the results
for the two sets of schools, subtracting the percentage Relative Change (keeping the sign) for the
comparison schools from that of the intervention schools. If this strategy is followed, the Rela-
tive Change becomes 28.3% for being bullied (23.6 + 4.7) and 58% for bullying other students
(21.4 + 36.6).
Considering the results for this study, it should also be emphasized that the OBPP had been in
place for only 6 months or less when the second measurement took place. In addition, this par-
ticular year was a very turbulent one for all Norwegian teachers with the introduction of a new
National Curriculum that made heavy demands on their time and emotional resources. is is
likely to have reduced the quality of implementation of the program in this study.
e (First) Oslo Project against Bullying is project (previously, in Olweus, 2004, called the
Oslo Project agai nst Bullying; Olweu s, 2001b, 2005) used the ordinary ex tended selection cohorts
388 • Dan Olweus and Susan P. Limber
design with two measurements, separated by a 1 year interval (November 1999–November
2000). For the approximately 900 students (at both time points) in grades 5 through 7, there
were marked reductions in bully/victim problems: For girls and boys combined, the Relative
Change (reduction) was 42.7 % (from 14.4% to 8.3%) for being bullied and 51.6% for bullying
other students (from 66% to 3.1%). e results are shown in Figures 27.1 and 27.2.
e Need for Evidence-Based Intervention Programs
As bully/victim problems have gradually been placed on the o cial school agenda in many
countries, a number of suggestions about their handling and prevention have been proposed.
Some of these suggestions and approaches seem ill-conceived or maybe even counter-produc-
tive, others appear meaningful and potentially useful. A key problem, however, is that most of
them have either failed to document positive results or have never been subjected to systematic
research evaluation. erefore it is di cult to know which programs or measures actually work
and which do not.
e situation is well illustrated by the following facts. From the mid-1990s, a U.S. expert
committee—the Blueprint Committee—has been engaged in systematically evaluating more
than 600 presumably violence (or generally problem-behavior) preventing programs. For a pro-
gr am to be approved by t he committe e, it must f ul ll certain minimum-level criteria (see Elliott,
1999):
at the program had produced positive e ects on relevant target groups (students in this •
case) in a relatively rigorous scienti c evaluation.
at the e ects had lasted for at least 1 year. •
at the program had produced positive results in at least one site beyond the original one. •
Figure 27.1
e Olweus Bullying Prevention Program • 389
Up to 2008, only 11 of the evaluated programs (four of which are school-based and only
one focusing on bullying) have satis ed the speci ed criteria and have been named “Blueprint
Programs.” A similar evaluation by an o cially appointed, departmental committee was made
in Norway in 2000. In this case, 25 programs designed to counteract and/or prevent “problem
behavior” and in use in Norwegian schools were evaluated (Rapport, 2000). Only one program
was recommended for further use in Norwegian schools without reservations.
e fact that the OBPP is one of the 11 Blueprint programs (Olweus et al., 1999) and, maybe in
particular, was the program selected by the Norwegian departmental committee, is an impor-
tant background for a government-funded national initiative in Norway.
A New National Initiative Against Bullying in Norway
In 2001, the Department of Education and Research (UFD) and the Department of Children and
Family A airs (BFD) decided that the OBPP was to be o ered on a large-scale basis to Norwe-
gian comprehensive schools over a period of years. In the period from 2001 to 2008, the program
has been implemented in approximately 500 schools and more than 125 OBPP instructors have
received thorough training in use of the program.
From an evaluation perspective, the new national initiative has provided a unique oppor-
tunity to examine the e ects of the OBPP on very large samples of students and schools under
ordinary conditions (i.e., without special e orts in terms of sta input or other resources) in the
context of large-scale dissemination (Flay et al., 2005).
When a new school or group of schools adopt the OBPP, they regularly start by administering
the Olweus Bullying Questionnaire (Olweus, 1996, 2007) some months before introduction of
the program. One year a er the initial measurement, the schools conduct a new survey with the
same questionnaire to nd out what results have been obtained and in what ways the individual
Figure 27.2
390 • Dan Olweus and Susan P. Limber
school may need to increase its e orts. Many schools make new assessments a er one, two, or
more additional years.
By having data from several consecutive cohorts of schools entering the OBPP at half-year
intervals, an excellent opportunity is provided to evaluate the reasonableness of a “history
interpretation” of obtained intervention ndings. As mentioned, such an interpretation implies
that the results could be a consequence of general time trends, special media attention or some
other “irrelevant” factor concomitant to the intervention rather than intervention. In particu-
lar, in September 2002, several important public actors including the Norwegian government,
Children’s Ombudsman, and the Teachers’ Union o cially signed a Manifesto against Bullying
which gained a good deal of media attention. And at the end of 2002, both the King of Norway
and the Prime Minister discussed the importance of counteracting bullying in schools in their
traditional television and radio talks on New Year’s Eve. In other phases of the evaluation period,
there was only sporadic media attention to bully/victim problems in school. If such manifesto
declarations and accompanying media attention had an e ect on the amount of bully/victim
problems in the country, this would be expected to show up in reduced levels of problems in
schools entering the OBPP immediately or relatively shortly a er the occurrence of these events.
e extent to which empirical data support this assumption will be examined below.
In the research evaluations, we analyzed the data using the extended selection cohorts design,
in which developmental or maturational e ects are controlled, as described above. In addition,
it is worth noting that the questionnaire data were collected by the schools themselves (follow-
ing detailed written instructions about the administration procedures) and that the author of
the OBPP did not participate other than sporadically in the training of instructors and had no
contact whatsoever with the participating schools. Almost all of the training was undertaken by
specially trained colleagues from the Olweus Group against Bullying at the Research Center for
Health Promotion, University of Bergen.
Major Results from the New National Initiative Project. Key results are shown in Figures
27.3 and 27.4. e upper curve of Figure 27.3 portrays the baseline percentages of bullied
students in grades 4 through 7 from ve di erent cohorts of schools which conducted their
introductory surveys in the period from October 2001 to October 2003. As in earlier studies,
to be classi ed as being bullied students had to respond to the global question in the Olweus
Bullying Questionnaire that they had been bullied “2 or 3 times a month” or more in the past
couple of months.
So far, we have mainly focused our evaluation analyses on the rst three of these cohorts.
e percentages of students (in the same grades and schools) who reported being bullied one
year later, when the schools had used the OBPP for approximately 8 months, are shown in the
lower curve of Figure 27.3. e two data points from each cohort of schools are connected with
an arrow. To illustrate, the percentage of bullied students in the rst cohort of schools (n = 8,388
students from 56 schools) was 15.2% while at follow-up 1 year later (n = 8299) this percentage
had been reduced to 10.2%, a relative reduction of 33%. e relative reductions for the two con-
secutive cohorts of schools were almost identical, both amounting to approximately 34%, from
14.0% (n = 4083 students from 46 schools) to 9.2% (n = 4,089) for the second cohort, and from
13.2% (n = 8,238 students from 58 schools) to 8.7% (n = 8,483) for the third cohort. Absolute
reductions for these three cohorts amounted to 4.9, 4.8, and 4.5 percentage points, respectively.
In Figure 27.4, the variable portrayed is bullying other students (“2 or 3 times a month” or
more in the past couple of months). e general pattern of results is very similar to what was
reported for being bullied in Figure 27.3, but at a lower level. e relative reductions for this
variable for the three cohorts of schools were 37% (from 5.7% to 3.6%), 48% (from 5.9% to 3.1%),
e Olweus Bullying Prevention Program • 391
and 49% (from 5.1% to 2.6%), respectively. e absolute reductions amounted to 2.1., 2.8, and
2.5 percentage points.
Figures 27.3 and 27.4 show percentages for boys and girls combined for grades 4 through 7.
Similar results were obtained when the data were analyzed separately for the two genders, the
four grades, and when an even stricter criterion (“about once a week” or more o en) was used
in classifying students as being bullied or bullying other students. Marked improvements could
thus be registered also for students who had been involved in more frequent and serious bully/
victim problems. (It should be noted that having been bullied/bullied other students “2 or 3
8
10
12
14
16
Oct 01 May 02 Oct 02 May 03 Oct 03
Baseline data Afte r 8 mo nths with the Olweus Program
Figure 27.3 Percentage bullied students 2001–2003, elementary grades 4–7 before (upper curve) and after (lower curve)
intervention. (Olweus, 2005, 11, 389-402). Reproduced with permission.
2
3
4
5
6
7
Oct 01 May 02 Oct 02 May 03 Oct 03
Baseline data After 8 months with the Olweus Program
Figure 27.4 P erc entage bul lying s tudents 2 001–2 003, elem enta ry grad es 4 –7 be fore (upper cur ve) and after (low er curv e)
intervention. (Olweus, 2005, 11, 389-402). Reproduced with permission
392 • Dan Olweus and Susan P. Limber
times a month” by no means represents non-serious or trivial problems, as shown in Solberg and
Olweus, 2003.) Similar positive results were also obtained with regard to the type of bullying
that girls are o en involved in—social isolation, rumour-spreading (relational bullying).
It is worth noting that the absolute percentage reductions in bully/victim problems in these
analyses are roughly similar to those obtained in the First Bergen Project against Bullying (p.
378), although the relative reductions in the latter project are somewhat larger due to lower
baseline (Time 1) values. Generally, it is important to emphasize that, although the results for
the three cohorts of schools were strikingly similar, these studies are quite independent and are
regarded as three separate evaluations/replications of the OBPP.
Is a “History Interpretation” Reasonable? As mentioned above, when using a selection
cohorts design, a history interpretation of the results should be considered a possible threat to
the validity of the conclusions and ideally be eliminated or made unlikely. Since the schools in
our three studies came from many di erent parts of Norway, it is natural to look for general time
trends or events with considerable media attention that might have an e ect across the whole
country as possible explanatory factors.
If we look at the upper curves in Figures 27.3 and 27.4 re ecting the levels of bully/victim
problems in the ve consecutive cohorts of schools that had not yet started with the OBPP, we
can observe a slight decline over time. But there are no marked breaks in the curves around the
time period—the latter half of 2002 and the beginning of 2003—when the Manifesto against
Bullying was signed and the Prime Minister and the King held their o cial New Year’s speeches.
If these media events had had a clear general e ect, we would have expected the levels of bully/
victim problems of the schools administering the questionnaire in October 2002 and possibly
May 2003 to have been considerably lower than those of the surrounding cohorts of schools.
Further, if there were such general e ects, one would expect a relatively marked di erence in
the relative reductions in bully/victim problems for the various cohorts of schools. In particular,
such possible e ects would very likely have a ected the levels of bully/victim problems of the
rst cohort of schools at the Time 2 assessment that was made in October 2002 (lower curve).
One would thus have expected that the relative percentage reduction for this cohort would have
been greater than for the other two consecutive cohorts. As can be seen in the gures, this was
not the case. Accordingly, the pattern of data obtained certainly does not suggest that media
attention and associated events had any observable e ect on the levels of bully/victim problems
of consideration here. However, the signing of the Manifesto and the related media attention
seemed to have an e ect on the schools/ communities’ interest and willingness to apply for par-
ticipation in the OBPP. e number of schools that took part in the OBPP training during the
fall/spring period of 2002/2003 almost doubled (n = 220) compared to the corresponding period
one year before (n = 119).
With regard to possible general time trends, the very similar results from successive, roughly
comparable cohorts of schools (without intervention) indicate considerable stability over time
in the average amount of bully/victim problems and do not suggest any marked time trend
uctuations in these levels. We have also found similar levels of stability when individual
schools or groups of schools have been followed over time (typically at yearly intervals) without
intervention.
In this context, it may be mentioned that the respected statistician Charles Reichardt in sev-
eral places speaks quite favorably about the (simple) selection cohorts design for the evaluation
of intervention e ects in his chapter in Cook and Campbell’s Quasi-experimentation: Design
and Analysis Issues for Field Settings (1979, pp. 198–199). Also, in the new and revised edition of
e Olweus Bullying Prevention Program • 393
this book, Shadish, Cook and Campbell (2004, pp. 151–153) suggest certain ways of strengthen-
ing the (simple) selection cohorts design by including additional cohorts. is is clearly in line
with the thinking behind use of the extended version of the selection cohorts design (above, and
Olweus, 1991, 2005). In this form, the selection cohorts design with extensions both with regard
to grade levels and number of consecutive cohorts is no doubt a quite strong design for e ect
evaluation.
A 5-Year Follow-Up Study in Oslo. For 14 of the 24 Oslo schools from the rst cohort, we
have data for ve assessments, from October 2001 to October 2006. e starting level for being
bullied was 14.0%, which decreased to 9.8% the following year. e reduced level was retained
and even further slightly reduced in the following years, reaching 8.4% in 2006. is change
represents a 40% relative reduction from the starting value.
For bullying other students, the 2001 value was 5.5%, which declined to 2.8% in 2002 and to
2.7% in 2006, a relative reduction of 50.9%. e number of students participating in these analy-
ses amounted to approximately 3,000 per assessment point. ese results are important, since it
has been shown (e.g., Beelman, P ngstein, & Lösel, 1994) that many program e ects are short-
lived and are found to be considerably reduced when longer-term e ects have been assessed
(even a er only 2 months a er the end of the program phase). e reported results show that the
e ects of the OBPP can be quite long-term and suggest that the intervention schools had been
able to change their “culture” and competence to counteract bullying in a more permanent way.
It should als o be noted that none of t he stude nts who took the quest ionnaire in 2001 par ticipated
in the 2006 assessment.
Norwegian Evaluation Studies Involving Grade 8–10 Students
Systematic use of the OBPP with students in grades 4 through 7 has consistently produced
positive results and this record seems to be relatively unique in an international perspective
(see Smith, Pepler, & Rigby, 1994; Tto & Farrington, 2009; Tto , Farrington, & Baldry, 2008).
Positive results have also been obtained with students from junior high/lower secondary school
grades (8 through 10), although less consistently and sometimes with weaker e ects. For exam-
ple, in the First Oslo Project, the OBPP was reasonably well implemented in Grade 8 and at Time
2 (in Grade 9) the percentage of bullied students was 6.3% compared with the 9.2% for the Grade
9 cohort 1 year earlier (at Time 1). is decrease represented a relative reduction by 32%. Also in
the First Bergen Project, positive results were obtained for the junior high school cohort (Grade
8), a er 8 and 20 months of intervention, respectively (see e.g., Olweus, 1991, 1997, where the
designation Grade 7 corresponds to Grade 8 in the United States and today’s Norwegian grade
system). In the context of the New National Initiative, however, results for these grades at the
1-year follow-up (Time 2) have been more variable, sometimes producing positive results, at
other times showing basically no di erence between the Time 1 and Time 2 assessments.
However, in a recent study following 14 schools with grades 8 and 9 over a 2-year period
from 2001 to 2003, the results at Time 3, a er 20 months of intervention, were clearly di erent
form those at Time 2 and Time 1. e percentage of bullied students had decreased from 8.0% at
Time 1 to 5.8% at Time 3, a relative reduction of 27.5%. Similar results were obtained for bully-
ing other students, with a relative reduction of 31%, from 6.1% at Time 1 to 4.2% at Time 3. e
reductions from Time 1 to Time 2 were quite small for both variables. Although these results
are based on a relatviely small number of schools, they suggest that it may take longer time to
achieve consistently good results in grades 8–10 than in lower grades. Other researchers in the
394 • Dan Olweus and Susan P. Limber
eld also have had the experience that it is more di cult to reduce bully/victim problems in
junior high/lower secondary grades (e.g., Smith & Sharp, 1994; Smith et al., this volume; Stevens
et al., 2000; Salmivalli, Kaukiainen & Voeten, 2005).
We think there are several reasons for this state of a airs. Very brie y, one reason has to do
with the individual development of the students, most of them entering puberty and a period
of increasing liberation from parental in uence and becoming more generally oppositional to
authorities, including teachers (Arnett, 1992). Probably even more important, the organization
of the teaching is di erent at this grade level (in Norway, and most Western countries as far as
we know). e teachers become more subject oriented and many of the social functions that are
typically taken care of by the homeroom teacher in lower grades are not ful lled by anyone. In
this way, the program can be expected to be less well implemented in these grades, which also
has been documented in preliminary analyses in the New National Initiative Project (Olweus
& Kallestad, in press). Accordingly, it is not unreasonable to expect weaker and less consistent
results.
We are now in the process of planning a new intervention project in Norway with a special
focus on schools with students in grades 8–10. A major task in that project will be to try to
ensure that the various program components are implemented in a more systematic way.
Outcome Studies in the United States
South Carolina e rst evaluation of the OBPP in the United States (Limber, Nation, Tracy,
Melton, & Flerx, 2004; Melton et al., 1998) was conducted in the mid-1990s involving data from
elementary a nd middle schools in six pr imarily ru ral school dist ricts. e dis tricts were organiz ed
into matched pairs based on geographic location and the demographics of the students. In each
pair, the schools in one district were selected to receive the OBPP (Group A), while the schools
in the other districts served as a comparison group for the rst year of the project (Group B).
During the rst year of the project, there were 11 Group A (intervention) schools and 28 Group
B (comparison) schools. Schools were not randomly assigned to groups. Although the project
continued during a second year, it was found that implementation of the program in this period
was inadequate (Limber et a l, 2004). Accordingly, we restrict analyses here to the rst year of the
project. is decision is also motivated by the fact that only analyses involv ing the rst two time
points provide a clear-cut intervention vs. control comparison.
Although school-level demographic data were not available to researchers, district-wide
demographic data indicated that the ethnicity of students ranged from 46% to 95% African
American, and from 4% to 53% White. In all but one school district, the percentage of students
receiving free or reduced lunches (a measure of poverty) ranged from 60% to 91%.
Data were analyzed as a pretest-posttest design with students in grades 4, 5, and 6 at Time 1
and followed up in grades 5, 6, and 7 one year later, at Time 2 (a er 7 months of program imple-
mentation in Group A schools). It was not feasible to secure the identity of the individual stu-
dents in the assessments which precluded adjustment of the outcome data for individual pretest
values. Accordingly, the results reported are likely to be conservative due to less than maximum
statistical power. At both Time 1 and Time 2, students completed an English-language version
of the original Olweus Bullying Questionnaire (Olweus, 1986). ey also completed a 42-ques-
tion measure developed by Olweus to assess other antisocial behavior (Bendixen & Olweus,
1999). Students indicated the frequency with which they had engaged in a variety of antisocial
behaviors (e.g., such as stealing money or other things, skipping school, and starting a ght with
another student) within the past 3-month period. From these questions, we developed 8 scales:
e Olweus Bullying Prevention Program • 395
the (Cronbach’s alpha = .81), vandalism (.72), violence (.69), delinquency (.90), substance abuse
(.79), school misbehavior (.81), school sanctions (.74), and group delinquency (.70). (See Melton
et al., 1998, for a more detailed description of the scales.)
Results indicated that the rst year of the program a ected students’ engagement in bullying
and other antisocial activities. ere were consistent and signi cant (p <.01 or 001) Time (Time
1 vs. Time 2) × Group (A vs. B) interactions for several indices of bullying others (frequency of
bullying other students in the last couple of months, frequency of bullying other students in the
last week at school, frequency of participation in group bullying of others). At Time 1, 23.6% of
students in Group A schools (n = 2025) indicated that they had bullied others several times or
more. A er 7 months of program implementation, this percentage dropped to 19.9%, a reduc-
tion of 15.7%. By comparison, at Time 1, 18.5% of students in Group B schools (n = 4,229) had
bullied others several times or more o en and a er 7 months this percentage had risen to 20.7%,
an increase of 11.9%. us, students in the intervention schools had an overall relative reduction
in bullying others of 27.6%. However, no signi cant changes were observed in the frequency
with which students reported being bullied
As expected, there was an increase over time in the frequency of self-reported antisocial
behavior among control (Group B) students, while for the intervention students, there was either
no increase or a slower rate of increase with regard to general delinquency (p < .01) and a num-
ber of the scales concerning vandalism (p < .05), school misbehavior (p < .001), and sanctions for
school misbehavior (p < .005). e program thus seemed to slow the age-related rate of increase
in students’ involvement in antisocial behavior.
It should be noted that positive e ects of the OBPP on antisocial behavior were also obtained
in the First Bergen Project described above (Olweus, 1991). ese results are not surprising given
that one can easily see bullying as a form of antisocial behavior (Olweus, 1993b).
Philadelphia Researchers have assessed the e e c t i v en e s s o f t h e OB P P i n i n n e r - ci t y P h i l a d el p h i a
schools using the extended selection cohorts design. Black and Jackson (2007) examined the
e ectiveness of the program in six large public elementary and middle schools (enrollments
of 456–1,295 students) over the course of 4 years of implementation. Students were primarily
from low-income families (67%) and were predominantly African American (82%) and Latino
(10%). e evaluation measures included an observation instrument to assess Bullying Incident
Density (BID), the Olweus Bully/Victim Questionnaire (administered to students in grades
3–8), and a measure of delity of implementation.
e observation instrument consisted of a checklist of bullying behaviors that assessed
physical, verbal, and emotional bullying (including name-calling, hitting, pushing, inappro-
priate touching, rumors, spitting, relational exclusion, teasing, taunting, cursing, raising voice
in anger, and threatening gestures). Observations of middle school students took place during
lunch, while observations of elementary students took place at recess. ese areas were selected
because they were identi ed through the anonymous questionnaire as being “hot spots” for
bullying at the school. Fidelity of implementation was assessed yearly for each of 14 core com-
ponents of the program (as identi ed by Olweus, Limber, & Mihalic, 1999). Implementation of
each program component was dichotomous (positive or negative). Total delity scores for each
school were calculated as the total number of core components implemented divided by the total
number of components.
At baseline, incident density was 65 incidents per 100 student hours. A er 4 years, BID
had decreased 45% to 36 incidents per 100 student hours. ere was no signi cant correlation
between overall program delity of implementation and changes in BID. However, the authors
396 • Dan Olweus and Susan P. Limber
reported that those program components that were most strongly associated with decreased
BID included the posting of school rules about bullying, consistent enforcement of positive and
negative consequences, and training adult monitors to engage students in activities. Anony-
mous self-reports of being bullied varied by school, but ranged from a decrease of 10% to an
increase of 7% from Time 1 to Time 4. Unfortunately, at Time 4, only 1598 students completed
the questionnaire, compared with 3,741 at baseline. Due to the great attrition, rm conclusions
about students’ self-reported victimization cannot be drawn in this study.
Washin gt on Bauer and colle agues (2007) assessed the e e c ti ve ne s s o f t h e O BP P i n W as h in g to n
state, using a nonrandomized controlled trial with 10 public middle schools (grades 6–8, with
7 intervention and 3 control sites). White students represented the most prevalent ethnic group
in intervention schools (40%), followed by Asian students (24%), African American students
(12%), and Hispanic/Latino students (7%). Ethnic make-up of control schools was somewhat
di erent, with signi cantly fewer White students (23%) and more African American students
(28%). Researchers assessed relational and physical measures of victimization using four speci c
questions from the Olweus Bullying Questionnaire and found that relational victimization
decreased by 28% among White students in intervention schools, relative to their peers in
control schools, and physical victimization decreased by 37%. However, there were no similar
e ects for students of other races/ethnicities and no overall program e ects regarding rates
of victimization. Students in intervention schools were signi cantly more likely than those in
control schools to perceive other students as actively intervening in bullying incidents. Similar
analyses for student perceptions of teachers’ or other adults’ readiness to intervene were not
statistically signi cant. e authors concluded that “implementation of the OBPP—may lead to
variable di erences in e ectiveness based on factors related to culture, race, and the in uence
of the family/home environment” (p. 273) and recommended that school sta “should be aware
of the in uence that home, culture, and society have on student behavior, and tailor preventive
measures accordingly” (p. 273).
In this context, it is worth noting that implementation of the OBPP has met with success in
several ethnically diverse settings in the United States (e.g., Philadelphia, rural South Carolina,
and southern California). Nevertheless, the authors’ recommendations to carefully consider the
unique setting of implementation and to make necessary cultural adaptations is important and
is, in fact, encouraged by authors of the OBPP (Olweus et al., 2007).
Bauer and colleagues (2007) encourage continued use of the OBPP, as it “is the only bully-
ing prevention program that is available that is comprehensive and that encompasses a whole-
school approach” (p. 273). Further, they note that the program is an important vehicle for change
because it helps to establish a common language about bullying and provides schools with the
necessary framework to address bullying. ey comment that the implementation of the OBPP
in intervention schools was “broad and encompassed a signi cant regularity and consistency”
(p. 273) that was lacking in comparison schools.
California Pagliocca and colleagues (2007) evaluated the e ectiveness of the OBPP in three
elementary schools in a suburban southern California community over a 2-year period using
a selection cohorts design. Outcome measures included student self-reports on the Olweus
Bullying Questionnaire (N = 1174 at Time 1, N = 1085 at Time 2, and N = 1119 at Time 3),
and anonymous reports by parents (N = 761 at Time 1, N = 817 at Time 2, and N = 411 at
Time 3), and teachers (N = 100 at Time 1, N = 72 at Time 2, and N = 78 at Time 3). Overall,
self-reported rates of being bullied among students decreased 21% a er 1 year and 14% a er 2
years. Decreases were particularly marked among fourth graders, where researchers observed a
e Olweus Bullying Prevention Program • 397
decrease of 32% a er 1 year and 20% a er 2 years of implementation. Researchers also observed
decreases overall in students’ reports of bullying others a er 1 year (by 8%) and a er 2 years
(17%) of implementation.
Other ndings of note included increases in bullied students’ propensities to tell a teacher
about their experiences (Time 1 vs. Time 3), students’ perceptions that teachers or other adults
at school try to stop bullying (Time 1 vs. Time 2), teachers’ perceptions that there were clear
rules about bullying (67% increase between Time 1 and Time 3), and teachers perceptions that
they knew how to respond to bullying that they observed or heard about (78% increase between
Time 1 and Time 3). ere also were marked increases in teachers’ beliefs that bullying policies
had been fairly well or extremely well communicated to students (97% increase), parents (91%
increase), teaching sta (72% increase), and non-teaching sta (79% increase) between Time 1
and Time 3. Fi nal ly, pa rent s wer e more l ikely to b elieve t hat sc hool adm inistr ator s had done ver y
much to stop bullying (18% increase between Time 1 and Time 2) but there were no di erences
in parents’ perceptions of teacher activity to stop bullying.
Summary of U.S. Findings To date, studies have evaluated the e ectiveness of the OBPP in
several diverse settings and elementary and middle school populations in the United States.
Some have been conducted by authors of the OBPP (Limber et al., 2004; Pagliocca et al., 2007),
while others have not (Bauer et al., 2007; Black & Jackson, 2007). e picture that emerges from
these studies is that the OBPP has had a noticeable impact on students as well as adults. Clear
decreases have been obser ved in students’ self-reported bully ing behavior, and antisocial
involvement (Limber et al., 2004; Melton et al., 1998; Olweus, 1991), victimization (Bauer et
al., 2007, for White students; Pagliocca et al., 2007), child victims’ propensities to report
bullying to adults at school (Pagliocca et al., 2007), and students’ perceptions that students
intervene to put a stop to bullying (Bauer et al., 2007). Observational measures of bullying
among students (measured as Bulling Incident Density) also have shown signi cant
decreases in relational and physical victimization (Black & Jackson, 2007) related to the
implementation of the OBPP. Finally, in the one study to assess adults’ perceptions of policies
and practices related to bullying, teachers perceived clear improvements in schools that
implemented the OBPP.
Although these ndings are clearly encouraging, it should also be noted that the results from
these studies have not been uniformly positive. Moreover, it would have been desirable to have
somewhat more knowledge about and control over the delity of implementation of the pro-
gram for some studies. Accordingly, current U.S. research (both planned and underway) will
involve more detailed analyses of the e ectiveness of the various program components and the
conditions under which the program has the largest e ects.
Conclusions and Practical Implications
e Olweus Bullying Prevention Program is built on the conviction that bullying need not and
should not be a common or “natural” experience for children and youth. Results from more than
20 years of research, primarily in Scandinavia and the United States, con rm that bullying can,
in fact, be considerably reduced through systematic school-wide e orts that reduce the oppor-
tunities and rewards for bullying and build a sense of community among students and adults.
Such a restructuring of the school environment does not come without considerable commit-
ment and e ort on the part of administrators, sta , students, and parents. However, when one
considers the numbers of students a ected and the tremendous personal and economic costs
of bullying—to involved students and their families, the broader school environment, and to
society at large—these e orts are not only reasonable but quite necessary.
398 • Dan Olweus and Susan P. Limber
e numbers of students who may have avoided direct involvement in bullying (as victims
or perpetrators of bullying) as a consequence of the OBPP is substantial. To illustrate, it may
be useful to focus on the Norwegian ndings. If we combine all six Norwegian large-scale
evaluation studies presented in this chapter, there are some 25,000 students who have par-
ticipated in the 1-year evaluations for grades 4–7 (taken the questionnaire at two time points
separated by a year). If we assume conservatively, on the basis of our empirical results, that
about 4% of the students in these grades have escaped being regularly bullied during the evalu-
ation year, this means that a considerable number of students—approximately 1,000—have
had safer and more positive school experiences for much of the evaluation period. We can also
ma ke the re asonable as sump tion that po siti ve e ects have been obtained for the approximately
25,000 students in grades K-3 who attended the same schools but did not participate in the
questionnaire assessment. Assuming similar e ects in these grades, the number of students
who escaped bullying through the intervention program increases to approximately 2,000
students in these schools.
We can further assume that a certain proportion of the intervention schools have been able
to maintain reduced levels of bully/victim problems also a er the introductory implementa-
tion period is over, as was found in the 5-year follow-up Oslo project. In such a perspective, it
becomes obvious that the e ect of the intervention program in terms of numbers of students
who have escaped bullying across multiple years, is quite substantial.
In suggesting an economic interpretation of these results, it is natural to point out that, with
overwhelming probability, they represent very signi cant savings for society with respect to psy-
chological/psychiatric treatment and health-related costs. ere is ample documentation that a
considerable proportion of victims of bullying su er from depression, anxiety, poor self-esteem,
and suicidal thoughts (e.g., Hawker & Boulton, 2000; Solberg & Olweus, 2003). A similar pattern
of negative e ects was also evident in a prospective follow-up study of two groups of boys aged
24 who had or had not been regularly bullied in school in grades 7 through 10, some 9 years ear-
lier (Olweus, 1993b). A recent Norwegian thesis has further documented that among 160 young
adults who sought psychiatric treatment for the rst time (at an average age of 35 years), some
50% had been bullied during their school years, and the more they had been bullied, the greater
their psychiatric symptoms (Fosse, 2006).
Although we have not (yet) assessed direct e ects of the OBPP on academic achievement, it
is ver y reasonable to assume that reductions in bullying would lead to increases in achievement,
particularly for victims of bullying but also more generally, for classrooms with bully/victim
problems. In the First Bergen Project, for example, the program e ects included clear improve-
ments in several “social climate” dimensions very likely related to academic achievement, as
mentioned above. In a recent longitudinal study conducted in the United States, Buhs, Ladd,
and Herald (2006) observed that peer rejection in kindergarten was associated with peer exclu-
sion in grades K-5 (e.g., excluded from activities), which in turn was associated with decreased
classroom participation, and ultimately lower academic achievement.
Although it has been found that former male school bullies are clearly overrepresented in
the crime registers as young adults (Olweus, 1993; Sourander et al., 2007), it has not (yet) been
documented that the OBPP directly leads to a reduction of adult criminality. However, in both
the First Bergen Project and the South Carolina project, it was shown that the program also had
clear e ects on concurrent antisocial behaviors such as vandalism, the , and truancy. Accord-
ingly, it is very reasonable to assume that at least some proportion of the students who stop
bullying in school as a consequence of the OBPP will be de ected from an antisocial trajectory.
Considering the very major costs imposed by individuals with conduct problems or conduct
disorder (e.g., Cohen, 1998; Scott, Knapp, Henderson, & Maughan, 2001), such a result would
e Olweus Bullying Prevention Program • 399
represent a very substantial saving to society, even if the number of “socialized” or averted bul-
lies were quite small.
Ongoing evaluations of the OBPP in Norway, the United States, and elsewhere around the
globe are being undertaken to assess the e ectiveness of the program in diverse contexts and
populations, which program components are particularly critical to program success, and
which teacher-, school-, and community-level variables are particularly important with regard
to program implementation. Assessing the general and di erential e ects of the OBPP on chil-
dren’s psychosocial well-being and mental health, academic achievement, and involvement with
antisocial peers and the criminal justice system also will be very useful in order to estimate the
potential savings that society can expect from the prevention of bully/victim problems in school
and elsewhere.
References
Arnett, J. (1992). Reckless behavior in adolescence: A developmental perspect ive. Developmental Review, 12, 339–373.
Bauer, N., Lozano, P., & Rivara, F. P. (2007). e e ectiveness of the Olweus Bullying Prevention Program in public
middle schools: A controlled trial. Journal of Adolescent Health, 40, 266–274.
Baumrind, D. (1967). Child care practices anteceding three patterns of preschool behavior. Genetic Psychology Mono-
graphs, 75, 43–88.
Beelma nn, A., P ngsten, U., & Lösel, F. (1994). e e ects of training social competence in children: A meta-analysis
of recent evaluation studies. Journal of Clinical Child Psychology, 23, 260–271.
Bendixen, M., & Olweus, D. (1999). Measurement of antisocial behaviour in early adolescence: Psychometric proper-
ties and substantive ndings. Criminal Behaviour and Mental Health, 9, 323–354.
Biglan, A. (1995). Changing cultural practices: A contextualist framework for intervention research. Reno, NV: Context
Press.
Black, S. A., & Jackson, E. (2007). Using bullying incident densit y to eva luate the Olweus Bullying Prevention Pro-
gramme. School Psychology International, 28, 623–638.
Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchic al linear models: Applications and data analysis methods. Newbur y
Park, CA: Sage.
Buhs, E. S., Ladd, G. W., & Herald, S. L . (2006). Peer exclusion and victimization: Processes that mediate the relation
between peer group rejection and children’s classroom engagement and achievement? Journal of Educational
Psychology, 98, 1–13.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic.
Cohen, M. A. (1998). e monetary value of saving a high-risk youth. Journal of Quantitative Criminology, 14, 5–33.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for eld settings. Chicago:
Rand McNally.
Craig, W. M., & Harel, Y. (2004). Bullying, ghting and victimization. In C. Curry, C. Roberts, A. Morgan, R. Smith,
W. Settertobulte, O. Samdal, et a l. (Eds.), Young people’s health in context. WHO report no. 4, Health Policy for
Children and Adolescents (pp. 133–144). Geneva: World Health Organization.
Elliott, D. S. (1999). Editor’s introduction. In D. Olweus, S. P. Limber, & S. Mihalic, (1999). e Bullying Prevention
Program: Blueprints for Violence Prevention, 9 (pp. xi–xxiii). Boulder, CO: Center for the Study and Prevention
of Violence.
Eslea, M., & Smith, P.K. (1998). e long-term e ectiveness of ant-bullying work in primary schools. Educational
Research, 40, 203–218.
Flay, B. R., Biglan, A., Gonzalez Cast ro, F., Gottfredson, D., Kellam, S., Moscicki, E. K., et al. (2005). Standa rds of evi-
dence: Criteria for e cacy, e ectiveness and dissemination. Prevention Science, 6, 151–175.
Fleiss, J. L. (1994). Measures of e ect size for categorical data. In H. Cooper & L. V. Hedges (Eds.), e Handbook of
Research Synthesis (pp. 245–260). New York: Russell Sage.
Fosse, G. K. (2006). Mental health of psychiatric outpatients bullied in childhood. Doctoral thesis, Department of Neu-
roscience, Faculty of Medicine, Norwegian University of Science and Technology, Trondheim.
Furre, H., Danielsen, I.-J., Stiberg-Jamt, R., & Skaalvik, E. M. (2006). Analyse av den nasjonale undersøkelsen
”Elevundersøkelsen”[Analysis of the national study ” e Student Investigation”]. Kristiansand, Norway: Oxford
Research.
Hanewinkel, R . (2004). Prevention of bully ing in German schools: An evaluation of an anti-bullying approach. In P.
K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 81–97). Ca m-
bridge, UK: Cambridge Universit y Press.
400 • Dan Olweus and Susan P. Limber
Hawker, D. S. J., & Boulton, M. J. (2000). Twenty years’ research on peer victimization and psychosocial maladjust-
ment: A meta-analytic review of cross-sectiona l studies. Journal of Child Psychology and Psychiatry and Allied
Disciplines, 41, 441–455.
Judd, C. M., & Kenny, D. A. (1981). Estimating the e ects of social interventions. New York: Cambridge University
Press.
Kallestad, J. H ., & Olweus, D. (2003). Predicting te achers’ and school’s implementation of the Olweu s Bullying Pre ven-
tion Program: A multilevel study. Prevention & Treatment, 6, Article 21, posted October 1, 2003, at http://www.
journals.apa.org/prevention
Kish, L. (1987). Statistical design for research. New York: Wiley.
Kjærnsli, M. Lie, S., Olsen, R. V., Roe, A., & Turmo, A. (2004). Rett spor eller ville veier? [On the right or the wrong
track?]. Oslo: Universitetsforlaget.
Kjærnsli , M. Lie, S., Olsen, R. V., Roe, A., & Turmo, A. (200 6). Norwegia n reports from TI MSS and PISA 2003: Summar y
and conclusions. Oslo: Institute for Teacher Education and School Development, University of Oslo.
Kyriakides, L., Kaloy irou, C ., & Lindsay, G. (2006). An analysis of the Revised Olweus Bully/Victim Questionnaire
using the Rasch measurement model. British Journal of Educational Psychology, 76, 781–801.
Limber, S. P. (2004). Implementation of the Olweus Bullying Prevention Program in American Schools: Lessons
learned in the eld. In D. L. Espelage & S. M. Swearer (Eds.), Bullying in American schools: A social-ecological
perspective on prevention and intervention (pp. 351–363). Mahwah, NJ: Erlbaum.
Limber, S. P. (2006). e Olweus Bully ing Prevention Program: An overview of its implementation and research basis.
In S. Jimerson & M. Furlong (Eds.), Handbook of school violence and school safety: From research to practice (pp.
293–307). Mahwa h, NJ: Erlbaum.
Limber, S. P., Nation, M., Tracy, A. J., Melton, G. B., & Flerx, V. (2004). Implementat ion of the Oweus Bully ing Preven-
tion Program in the Southeastern United States. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools:
How successful can intervention s be? (pp. 55–79). Cambridge, UK: Cambridge University Press.
Loeber, R., & Stouthamer-Loeber, M. (1986). Family factors as correlates and predictors of conduct problems and
juvenile delinquency. In M. Tonry & N. Morris (Eds.), Crime and justice, 7 (pp. 219–339). Chicago: University of
Chicago Press.
Melton, G. B., Li mber, S. P. ,Cun ningham, P., Osgood, D. W., Chambers, J. Fler x, V., et al. (1998). Violence among rural
youth. Fina l Report. Washington, DC: U.S. Depart ment of Justice, O ce of Justice Progra ms, O ce of Juvenile
Justice and Delinquency Prevention.
Murray, D. (1998). Design and analysis of group-randomized tr ials. New York: Oxford University Press.
Olweus, D. (1973). Personality and aggression. I J. K. Cole, & D. D. Jensen (Eds.), Nebraska Symposium on Motivation
1972 (pp. 261–321). Lincoln: University of Nebraska Press.
Olweus, D. (1978). Aggression in the school s: Bullies and whipping boys. Washington, DC: Hemisphere.
Olweus, D. (1979). Stability of aggressive reaction patterns in males: A review. Psychological Bulletin, 86, 852–875.
Olweus, D. (1980). Familial and temperamental determinants of ag gressive behavior in adolescent boys: A causa l anal-
ysis. Developmental Psychology, 16 , 644–660.
Olweus, D. (1986). e Olweus Bully/Victim Questionnaire. Mimeo. Bergen, Norway: Research Center for Health Pro-
motion, University of Bergen.
Olweus, D. (1991). Bully/victim problems among schoolchildren: Basic facts and e ects of a school based interven-
tion program. In D. J. Pepler & K. H. Rubin (Eds.), e development and treatment of childhood aggression (pp.
411–448). Hillsdale, NJ: Erlbaum.
Olweus, D. (1993a). Bullying at school: What we know and what we can do. New York: Blackwell.
Olweus, D. (1993b). Victimization by peers: Antecedents and long-term outcomes. In K. H. Rubin & J. H. Asendort
(Eds.), Social withdrawal, inhibition, and shyness (pp. 315–341). Hillsdale, NJ: Erlbaum.
Olweus, D. (1996). e Revised Olweus Bully/Victim Questionnaire. Mimeo. Bergen, Norway: Research Center for
Health Promotion, University of Bergen.
Olweus, D. (1997). Bully/victim problems in school: Facts and intervention. European Journal of Psychology of Educa-
tion, 12, 495–510.
Olweus, D. (1999a). Norway. In P. K. Smith, Y. Morita, J. Ju nger-Tas, D. Olweus , R. Catalano, & P. Slee (Eds .), e nature
of school bullying: A cross-national perspective (pp. 28–48). London: Routledge.
Olweus, D. (1999b). Noen hovedresultater f ra Det Nye Bergensprosjektet mot mobbing og antisosial atferd [Some key
results from e New Bergen Project against bullying and Antisocia l Behavior]. Unpublished manuscript.
Research Center for Health Promotion, University of Bergen, Nor way.
Olweus, D. (2001a). Olweus’ core program against bullying and antisocial behavior: A teacher handbook. Bergen, Nor-
way: Author.
Olweus, D. (2001b). Antimobbningprojekt i Oslo-skolor med meget gode resultater [Anti-bullying project in Oslo
schools with very good results]. Unpublished manuscript. Research Center for Healt h Promotion, University of
Bergen, Norway.
e Olweus Bullying Prevention Program • 401
Olweus, D. (2004). Bullying at school: Prevalence estimation, a usef ul evaluation design, and a new nationa l initiative
in Norway. Association for Child Psychology and Psychiatry Occasional Papers, 23, 5–17.
Olweus, D. (2005). A useful evaluation design, and e ects of the Olweus Bullying Prevention Program. Psychology,
Crime & Law, 11, 389–402
Olweus, D. (2007). Olweus Bullying Questionnaire: Scannable paper version. Center City, MN: Hazelden.
Olweus, D., & Alsaker, F. D. (1991). Assessing change in a cohort longitudinal study with heirarchical data. In D.
Ma gnus son, L. R . Berg man , G. R udin ger, & B. Tores tad ( Eds .), Problems and method s in longitudinal re search (pp.
107–132). New York: Cambridge University Press.
Olweus, D., & Ka llestad, J. H. (in press). e Olweus Bullying Prevention Program: E ects of classroom components at
di erent grad levels. In K. Osterman (Ed.), Indirect and direct aggression. New York: Peter La ng.
Olweus, D., & Limber, S. P. (2007). Olweus Bullying Prevention Program Teacher Guide. Center City, MN: Hazelden.
Olweus, D., Limber, S. P., Flerx, V., Mullin, N., Riese, J., & Snyder. M. (2007). Olweus Bullying Prevention Program
Schoolwide Guide. Center City, MN: Hazelden.
Olweus, D., Limber, S. P., & Mihalic, S. (1999). e Bullying Prevention Program: Blueprints for Violence Prevention,
Vol .9. Boulder, CO: Center for the Study and Prevention of Violence.
Paglio cca, P. M., Limbe r, S . P., & Hash ima, P. (2007). Evalu ation report for the Chu la Vista Olweus Bul lying Prevention
Program. Chula Vista, CA: Chula Vista Police Depart ment.
Pepler, D. J., Craig, W. M., O’Connell, P., Atlas, R., & Charach, A. (2004). Mak ing a di erence in bullying: Evaluation
of a systemic s chool-based programme in Ca nada. In P. K. Smith, D. Pepler, & K. Rigby (Ed s.), Bullying in schools:
How successful can interventions be? (pp. 125–139). Cambridge, UK: Cambridge University Press.
Pepler, D. J., Craig, W., Ziegler, S., & Charach, A. (1994). An evaluation of an anti-bullying inter vention in Toronto
schools. Canadian Journal of Community Mental Health, 13, 95–110.
Rapport (2000). Vurdering av program og tiltak for å redusere problematferd og utvikle sosial kompetanse [Eva luat ion
of programs and measures to reduce problem behaviour and develop social competence]. Oslo, Norway: Kirke-,
undervisnings-, og forskningsdepartementet.
Roland, E. (1989). Bullying: e Scandinavian research tradition. In D. P. Tattum & D. A.Lane (Eds.), Bullying in
schools (pp. 21–32). Stoke-on-Trent, UK: Trentham Books.
Salmivalli, C., Kaukiainen, A., & Voeten, M. (2005). Antibully ing intervention: Implementation and outcome. British
Journal of Educational Psychology, 75, 465–487.
Scott, S., Knapp, M., Henderson, J., & Maughan, B. (2001). Financial cost of social exclusion: follow up study of antiso-
cial children in adulthood. British Medical Journal, 323, 191–194.
Shadish, W.R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental design for generalized
causal inference. Boston: Houghton-Mi in.
Smith, P. K., Pepler, D., & Rigby, K. (1994). Bullying in schools: How successful can interventions be? New York: Cam-
bridge University Press.
Smith, P. K., & Sharp, S. (Eds.). (1994). School bullying: Insights and perspectives. London: Rout ledge.
Smith, P. K., Sharp, S., Eslea, M., & ompson, D. (2004). Engla nd: the She eld project. In P. K. Smith, D. Pepler, & K.
Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 99–123). Ca mbridge, UK: Cambrid ge
University Press.
Solberg, M., & Olweus, D. (2003). Prevalence estimation of school bullying with the Olweus Bully/Victim Question-
naire. Aggressive Behavior, 29, 239–268.
Sourander, A., Jensen, P., Rönning, J. A., Elonheimo, H., Niemela, S., Helenius, H., et al. (2007). Childhood bullies
and victims and their risk of criminality in late adolescence. Archives of Pediatrics and Adolescent Medicine, 161,
546–552.
Stevens, V., De Bourdeaudhuij, I., & Van Oost, P. (2000). Bullying in Flemish schools: an evaluation of anti-bullying
intervention in primary and secondary schools. British Journal of Educational Psychology, 70, 195–210.
Stevens, V., Van Oost, P., & de Bourdeaudhuij, I. (2004). Inter ventions agains t bullying i n Flemish schools: prog ramme
development and evaluation. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in Schools: How Successf ul can
Interventions be? (pp. 141–165). New York: Cambridge University Press.
Tto , M. M., & Farrington, D. P. (2009). What works in preventing bullying: E ective elements of anti-bullying pro-
grammes. Journal of Aggression, Con ict and Peace Research, 1, 13–24.
Tto , M. M., Farrington, D. P., & Baldry, A. C. (2008). E ectiveness of programmes to reduce bullying. Stockholm:
Swedish National Council for Crime Prevention.
Whitney, I., Rivers, I., Smith, P., & Sharp, S. (1994). e She eld project: methodology and ndings. In P. Smith & S.
Sharp (Eds.), School bullying: Insights and perspectives (pp. 20–56). London: Routledge.
Zucker, D. (1990). An analysis of variance pitfall: e xed e ect analysis in a nested design. Educational and Psycho-
logical Measurement, 50, 731–738.
