ArticlePDF Available

Abstract and Figures

The purpose of this quasi-experimental study was to evaluate a large-scale implementation of the Olweus Bullying Prevention Program with children and youth in grades 3-11 in the U.S. Two major sets of analyses are presented, one following 210 schools over two years (Study 1; n = 70,998 at baseline) and the other following a subsample of 95 schools over three years (Study 2; n = 31,675 at baseline). Schools were located in 49 counties in central and western Pennsylvania. The Mplus 7.0 program was used to analyze the data which had a multilevel structure, with students nested in schools and program effects based on school-aggregated outcome variables. For almost all grades, there were clear reductions in the two key dimensions, being bullied and bullying other students. Average Absolute Change amounted to approximately 3%, implying that almost 2000 students had escaped being bullied in the two-year study. School-level Cohen's d's were large or fairly large. The longitudinal analyses documented increases in students' expressions of empathy with bullied peers, marked decreases in their willingness to join in bullying, and perceptions that their primary teacher had increased his or her efforts to address bullying. Overall, effects were stronger the longer the program had been in place. The analyses provided strong support for the effectiveness of the OBPP with U.S. students in elementary, middle, and early high school grades in Pennsylvania schools. Future research is warranted to assess program effectiveness in different racial/ethnic and community settings and to examine the relation between fidelity of implementation and outcomes.
Content may be subject to copyright.
Contents lists available at ScienceDirect
Journal of School Psychology
journal homepage: www.elsevier.com/locate/jschpsyc
Evaluation of the Olweus Bullying Prevention Program: A large
scale study of U.S. students in grades 311
Susan P. Limber
a,,1
, Dan Olweus
b,1
, Weijun Wang
c
, Matthew Masiello
d
,
Kyrre Breivik
e
a
Department of Youth, Family & Community Studies, 2038 Barre Hall, Clemson University, Clemson, SC 29634, United States
b
Psykologkonsult Dan Olweus, Vognstolbakken 16, NO-5096 Bergen, Norway
c
The State University of New York at Bualo, Research Institute on Addictions. University at Bualo, State University of New York, 1021 Main Street,
Bualo, NY 14203, United States
d
University of Massachusetts Memorial Children's Medical Center, Health Alliance Hospital, 60 Hospital Rd., Leominster, MA 01453, United States
e
Regional Centre for Child and Youth Mental Health and Child Welfare, Uni Research Health, Nygårdsgaten 112-114, 5008 Bergen, Norway
ARTICLE INFO
Action Editor: Michelle K. Demaray
Keywords:
Bullying victimization
Bullying perpetration
Evaluation
Olweus Bullying Prevention Program
Anti-bullying programs
ABSTRACT
The purpose of this quasi-experimental study was to evaluate a large-scale implementation of the
Olweus Bullying Prevention Program with children and youth in grades 311 in the U.S. Two
major sets of analyses are presented, one following 210 schools over two years (Study 1;
n= 70,998 at baseline) and the other following a subsample of 95 schools over three years
(Study 2; n= 31,675 at baseline). Schools were located in 49 counties in central and western
Pennsylvania. The Mplus 7.0 program was used to analyze the data which had a multilevel
structure, with students nested in schools and program eects based on school-aggregated out-
come variables. For almost all grades, there were clear reductions in the two key dimensions,
being bullied and bullying other students. Average Absolute Change amounted to approximately
3%, implying that almost 2000 students had escaped being bullied in the two-year study. School-
level Cohen's d's were large or fairly large. The longitudinal analyses documented increases in
students' expressions of empathy with bullied peers, marked decreases in their willingness to join
in bullying, and perceptions that their primary teacher had increased his or her eorts to address
bullying. Overall, eects were stronger the longer the program had been in place. The analyses
provided strong support for the eectiveness of the OBPP with U.S. students in elementary,
middle, and early high school grades in Pennsylvania schools. Future research is warranted to
assess program eectiveness in dierent racial/ethnic and community settings and to examine
the relation between delity of implementation and outcomes.
Bullying is an ancient phenomenon, yet systematic research on the nature and prevalence of bullying and eorts to prevent
bullying are relatively recent. The earliest study on bullying was conducted in the 1970s in Scandinavia (Olweus, 1973, 1978), and
the rst systematic attempts to prevent bullying also began in Scandinavia. In 1983, the Norwegian Ministry of Education launched a
nationwide campaign to address bullying in schools, in response to public concerns that were fueled by the suicides of three teenagers
who allegedly had been severely bullied by their peers. An initial version of what later became known as the Olweus Bullying
https://doi.org/10.1016/j.jsp.2018.04.004
Received 24 February 2017; Received in revised form 5 January 2018; Accepted 25 April 2018
Corresponding author.
1
The rst two authors contributed equally to this article.
E-mail addresses: slimber@clemson.edu (S.P. Limber), olweus@uni.no (D. Olweus), wwang@ria.bualo.edu (W. Wang),
mmasiello@healthalliance.com (M. Masiello), Kyrre.breivik@uni.no (K. Breivik).
Journal of School Psychology 69 (2018) 56–72
0022-4405/ © 2018 Published by Elsevier Ltd on behalf of Society for the Study of School Psychology.
T
Prevention Program (OBPP) was developed, implemented, and evaluated within this same context (Olweus, 1991, 1993;Olweus &
Limber, 2010b). Although international research on bullying grew slowly in the 1980s and 1990s, it was not until the late 1990s and
2000s that attention to bullying exploded among researchers, policy makers, and the general public in the U.S. and in many other
countries (Berger, 2007). Today, bullying is commonly recognized as a serious public health problem aecting children and ado-
lescents in the U.S. and around the world (Masiello & Schroeder, 2014;National Academies of Sciences, Engineering, and Medicine,
2016;Oce of the Special Representative of the Secretary-General on Violence against Children, 2016).
Bullying is a subset of aggressive behavior that is commonly repeated and involves a power imbalance between a target and his or
her perpetrator(s) (Gladden, Vivolo-Kantor, Hamburger, & Lumpkin, 2014;Olweus, 1993). In the U.S., nearly one-quarter of students
ages 1218 report having been bullied during the school year (Zhang, Musu-Gillette, & Oudekerk, 2016) and 14% of 3rd-12th graders
reported having been bullied 23 times per month or more (Luxenberg, Limber, & Olweus, 2015). The many short- and long-term
negative psychosocial, mental health, physiological, and behavioral eects of being bullied have been thoroughly documented and
will not be reiterated here (for overviews, see Cook, Williams, Guerra, Kim, & Sadek, 2010;National Academies of Sciences,
Engineering, and Medicine, 2016;Olweus, 2013;Tto, Farrington, Lösel, & Loeber, 2011a). Similarly, bullying others has been
associated with a number of short- and long-term negative characteristics, but of a dierent nature than those of individuals who
have been bullied (for overviews, see Cook et al., 2010;National Academies of Sciences, Engineering, and Medicine, 2016;Olweus,
2013;Tto, Farrington, & Lösel, 2012). Common consequences of being bullied by peers are largely internalizing, and include
depression, poor self-esteem, and suicidal ideation, whereas children and youth who bully peers are characterized by externalizing
problems, such as violence, rule breaking behavior, and delinquency.
As Olweus (1993) emphasized at an early stage, bullying is not only a health problem but also a serious violation of fundamental
human rights. In recognition of children's rights to be safe in school, and in response to concerns about the negative human and
societal eects of bullying, a number of school-based bullying prevention eorts have been launched in recent years (National
Academies of Sciences, Engineering, and Medicine, 2016). Several meta-analyses and/or systematic reviews of these prevention
programs have been conducted, with somewhat mixed results (for a review, see Tto, Eisner, & Bradshaw, 2014). The most com-
prehensive meta-analysis to date was conducted by Ttoand Farrington (2009, 2011). In this analysis, which included 44 evaluations
of school-based intervention programs, the authors concluded that anti-bullying programs were eective in reducing bullying and/or
victimization by an average of 1723% but that the eect sizes were relatively small. There was great variation in program eects,
however, and the authors noted that programs implemented in Europe were more eective than those implemented in the U.S. The
authors also observed that programs inspired by the work of Dan Olweus worked best(Tto& Farrington, 2011, pp. 4142).
Indeed, while the average Odds Ratio for all included studies was 1.36 for bullying perpetration and 1.29 for bullying victimization,
the (unweighted) average for evaluations of the Olweus Bullying Prevention Program was markedly higher, 1.83 and 1.80, re-
spectively (Tto& Farrington, 2011,Table 3, pp. 3637).
1. Brief description of the OBPP
The Olweus Bullying Prevention Program (OBPP; Olweus, 1991, 1993,Olweus & Limber, 2010a, 2010b) is the oldest and one of
the most researched bullying prevention programs in the world. It represents a whole-school comprehensive approach that includes
schoolwide, classroom, individual, and community components. The program is focused on both short-term and long-term change
that will create a safe and positive school environment. The overarching goals of the OBPP are to reduce existing bullying problems
among students, prevent new bullying problems, and achieve better peer relations (Olweus, 1993;Olweus & Limber, 2010b). These
goals are pursued by restructuring the school environment to reduce opportunities and rewards for bullying behavior and to build a
sense of community. The program was designed and initially evaluated for use in elementary, middle, and junior high schools.
The OBPP is built on four basic principles. Adults at school should: (a) show warmth and positive interest in students; (b) set rm
limits to unacceptable behavior; (c) use consistent positive consequences to acknowledge and reinforce appropriate behavior and
non-physical, non-hostile consequences when rules are broken; and (d) function as authorities and positive role models (Olweus,
1993;Olweus et al., 2007). These principles have been translated into interventions at the school, classroom, individual, and
community levels (Olweus, 1993;Olweus & Limber, 2010a, 2010b). Parent involvement is encouraged at all levels (Olweus & Limber,
2010b). There are eight school-level components, which are implemented school-wide, including the development of a Bullying
Prevention Coordinating Committee (BPCC, a building-level coordinating team of administrators, teachers, non-teaching sta, and
parents that is responsible for ensuring that all components of the OBPP are implemented with delity); the yearly administration of
the Olweus Bullying Questionnaire (OBQ, Olweus, 2007a); training and ongoing consultation for members of the coordinating team
and all school sta; the adoption of clear rules and policies about bullying; and the review and renement of the school's supervisory
system to reduce bullying. Classroom-level components of the program include holding regular class meetings to build understanding
of bullying and related issues through discussions and role play, and build class cohesion; posting and enforcing school-wide rules
against bullying; and holding periodic class-level meetings with parents. Teachers are also encouraged to integrate bullying pre-
vention messages and strategies throughout the curriculum. Individual-level components of the OBPP include supervision of students'
activities, particularly in known hot-spots for bullying; training for all stato help them intervene on-the-spot when bullying occurs
or is suspected; and follow-up interventions with children and youth involved in bullying. Finally, BPCC members are encouraged to
involve one or more community members on their team, look for ways that community members can support the school's program,
and collaborate to spread bullying prevention strategies and messages into other community settings that involve children and youth
(Olweus & Limber, 2010b).
School staare supported in their implementation by manuals for teachers and for members of the BPCC, class meeting resource
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
57
guides, and videos. Training and ongoing consultation are provided by certied OBPP Trainer-Consultants, who help schools address
challenges and maintain delity to the model. As described in detail elsewhere (Limber & Olweus, 2017;Olweus & Limber, 2010b),
although the basic principles and core components of the OBPP have remained largely unchanged since its development and initial
implementation in Norway, research and extensive experience have naturally led to some adaptations of the program to the U.S.
educational context, including the current implementation and evaluation. Such adaptations include development of English lan-
guage print and video resources for administrators, teachers, and parents; the organization of training of trainer-consultants and
school teams; and the use of readiness checklists.
1.1. Evaluations of the OBPP in Norway
The rst evaluation of the OBPP, which took place in Bergen, Norway, followed 2500 students in grades 58 in 42 schools over a
period of two-and-a-half years (between 1983 and 1985) (Olweus, 1991, 1993, 1997). Using an extended age cohort design
(Olweus, 2005, and below), in which same-aged students were compared across time, Olweus documented marked reductions in
students' self-reports of being bullied and bullying others, and signicant reductions in teachers' and students' ratings of bullying
among students within the classroom. In addition, there were signicant improvements in several measures of school/classroom
climate, including improvements in students' satisfaction with school life, improved order and discipline, more positive social re-
lationships, and a more positive attitude towards school (1991, 1993, 1997; see also summaries of ndings in Olweus & Limber,
2010a, 2010b). An indicator of delity of program implementation was signicantly related to program outcomes (Olweus &
Kallestad, 2010).
Subsequent to this initial study, six additional large-scale evaluations of the OBPP have been carried out in Norway, invol-
ving > 30,000 students from > 300 schools (Olweus, 2005;Olweus & Limber, 2010b). Findings have revealed consistently positive
program eects among students in grades 47, typically with reductions in bullying problems in the 3550% range after eight months
of intervention (Olweus & Limber, 2010b). Although positive ndings have also been obtained with students in grades 810, results
have been less consistent and it has taken longer to achieve as strong eects as with younger students (Olweus & Limber, 2010b). In
likely the rst study of long-term eects of a program, Olweus followed students from 14 schools in Oslo (with approximately 3000
students at each assessment) and observed reductions in self-reports of victimization of 40% and self-reported bullying of 51% over a
period of ve years (Olweus & Limber, 2010b).
1.2. Evaluations of the OBPP in the U.S.
The eectiveness of OBPP has been evaluated in two relatively large-scale studies in the U.S. The rst took place in the mid-1990s
and involved elementary and middle schools in six rural school districts in South Carolina with high percentages of African American
students (districts ranged from 46% to 95% African American; Limber, Nation, Tracy, Melton, & Flerx, 2004;Olweus & Limber,
2010b). After seven months of program implementation, signicant dierences between intervention and comparison schools were
observed with regard to students' reports of bullying other students, self-reported delinquency, vandalism, school misbehavior, and
sanctions for school misbehavior. There were no signicant program eects for students' reports of being bullied, however. Bauer,
Lozano, and Rivara (2007) used a nonrandomized controlled design to evaluate the OBPP with seven intervention and three control
schools in Washington State. Findings revealed signicant program eects for relational and physical victimization among white
students but not among students of other races/ethnic backgrounds (see Olweus & Limber, 2010b for more details). Although these
U.S. ndings are clearly encouraging, it is also clear that the results from these studies in the U.S. have not been uniformly positive.
The concern about the lack of positive program eects in the U.S. is echoed in a recent review of evaluations of bullying pre-
vention programs (Evans, Fraser, & Cotter, 2014), which was completed after the Ttoand Farrington meta-analysis. Unfortunately,
the conclusions from this review are somewhat compromised by the inclusion of four exceptionally small samples (n< 50) and
seven evaluations (out of 22) with < 200 participants. Still, it is worth noting that six of the eight studies examining bullying
victimization with nonsignicant results were conducted in the U.S., as were six of the ten nonsignicant studies examining bullying
perpetration. Several researchers have expressed doubt about the usefulness and likely success of bullying prevention eorts in
general, and such programs implemented outside of the U.S., in particular (e.g., Cohen, Espelage, Twemlow, Berkowitz, & Comer,
2015;Hong & Espelage, 2012).
In view of these concerns, there is obviously a marked need for a new large-scale study of the OBPP in the U.S., in which the
program has been systematically implemented over a period of at least two years and data from a sample of appropriate size have
been adequately analyzed with multilevel techniques taking account of cluster eects. The goal of the present article is to report the
results of such a study.
1.3. Research questions and expectations
The current study addressed several research questions pertaining to changes in students' behaviors and attitudes with respect to
bullying. Of primary interest was whether there could be documented systematic changes in students' reports of being bullied and
bullying other students after implementation and as a consequence of the OBPP.
We expected that there would be such reductions in being bullied and bullying others, that these eects would be observed for
both males and females and, tentatively (due to smaller sample sizes and a relatively large percentage of missing data on the relevant
variable, see Table 1 below) also for students of the races/ethnicities included in the study. In addition, and based on experiences
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
58
with OBPP evaluations in Norway (Olweus & Kallestad, 2010;Olweus & Limber, 2010b), we assumed eects for being bullied would
be somewhat stronger for elementary and middle school-aged students compared with high school students. Consistent with the
ndings of Olweus (1991, 2005) and Olweus and Limber (2010b), we expected that the eects would be stronger the longer the
implementation of the OBPP.
Not only did we expect to observe changes in students' bullying behaviors over time, but we also predicted changes in students'
attitudes towards bullying after being exposed to the OBPP. Specically, we anticipated that negative attitudes towards bullyingin
particular, expressing a disinclination to join in bullyingwould increase over time. Not surprisingly, research has found that at-
titudes towards bullying are strongly related to the likelihood of bullying others (Salmivalli & Voeten, 2004;Van Goethem, Scholte, &
Wiers, 2010). Moreover, a focus of the OBPP is on recognizing and altering the behavior of bystanders to bullyingthose who do not
initiate bullying but may assume a variety of roles upon observing bullying, ranging from joining in the bullying, to watching
passively, to helping to stop bullying (Olweus et al., 2007;Salmivalli, Lagerspetz, Björkqvist, Österman, & Kaukiainen, 1996). Thus,
we expected to see reductions in the extent to which students believed they could join in bullying over time, for boys and girls, and
across all age groups.
The current study also examined whether students expressed more empathy for bullied peers over time. A recent meta-analysis
has conrmed that empathy is negatively associated with bullying others and is positively associated with defending peers from
bullying (Zych, Tto, & Farrington, 2017). In addition, short-term longitudinal research has found that higher empathy predicted less
future involvement in bullying (Stavrindes, Georgiou, & Theofanous, 2010). Several components of the OBPPnamely class meeting
discussions and role playingare intended to build a sense of community among students and increase empathy for bullied peers
(Olweus & Limber, 2010b). Thus, we expected that over time, students over time would express more empathy for bullied peers.
Finally, we also expected that students would perceive increased anti-bullying activities by their teachers in their schools over
time. We anticipated that with the implementation of the OBPP, obvious changes in teachers' eorts to prevent and address bullying
would occur and that these behaviors would be apparent to students at all grade levels.
These research questions were addressed in two studies, which will be described in detail below. Study 1 addresses possible
program eects with regard to being bullied and bullying others for a large sample of students over two years, and assesses dif-
ferences in outcomes by gender and race/ethnicity. Study 2 examines longer-term eects for a sub-group of students involved in the
program. In this second study, we examined changes in students' involvement in bullying involvement (being bullied or bullying
others), attitudes towards bullying, empathy for bullied peers, and students' perceptions of teachers' actions to address bullying over a
period of three years.
2. Method
2.1. Participants
The original sample included students in grades 312 who were drawn from schools that were involved in a wide-scale eort to
implement the OBPP in elementary, middle, and high schools in 49 counties in southern and central Pennsylvania (Limber & Olweus,
2017). Our target population in this study consisted of students from schools that had completed successful standard implementation
Table 1
Characteristics of the participants (at baseline) from the Two-year and Three-year studies.
2-Year study 3-Year study
N(%) N(%)
Sex
Female 34,820 (49.0) 15,560 (49.2)
Male 35,862 (50.5) 15,937 (50.4)
Missing (sex) 316 (0.4) 123 (0.4)
Grade
Grade 3 8636 (12.2) 4447 (14.1)
Grade 4 8586 (12.1) 4402 (13.9)
Grade 5 9161 (12.9) 4446 (14.1)
Grade 6 11,397 (16.1) 4502 (14.2)
Grade 7 12,002 (16.9) 4156 (13.1)
Grade 8 11,913 (16.8) 4185 (13.2)
Grade 9 3404 (4.8) 1990 (6.3)
Grade 10 2844 (4.0) 1555 (4.9)
Grade 11 3055 (4.3) 1937 (6.1)
Race/ethnicity
White 44,028 (62.0) 19,291 (61.0)
Black or African American 3575 (5.0) 1432 (4.5)
Hispanic or Latino 3177 (4.5) 1006 (3.2)
Other 8165 (11,5) 3539 (11.2)
Missing data/don't know 12,053 (17.0) 6352 (20.1)
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
59
of the OBPP, as determined by the minimum criterion that the school had conducted the Olweus Bullying Questionnaire (Olweus,
2007a, 2007b) at baseline and two years later. Although completion of these two surveys does not in itself represent a detailed
assessment of the degree of delity of program implementation, the surveys are important components of the program and thereby
provide a rough indication that the program had been introduced in the school in a standardized way, as recommended in the guides
for program leaders (Olweus et al., 2007) and teachers (Olweus & Limber, 2007). Accordingly, 20 of the original 230 schools were
removed from the analyses because they had conducted only one survey. Twelfth grade students were also omitted from the analyses
because their numbers were much lower than those in other high school grades (approximately 50% of the numbers of students in
grades 911). The resulting sample (Study 1) thus included participants who were students in grades 311, drawn from 210 schools,
and representing three dierent cohorts that began to implement the program in 2008 (83 schools), 2009 (103 schools), and 2010 (24
schools). Locales for the 210 schools were primarily suburban (59%), and rural (29%), with 9% of the schools located in urban
locales, and 4% located in towns (National Center for Education Statistics [NCES], 2014). A total of 70,998 students completed
baseline (T0) assessments prior to implementation of the OBPP, and 67,374 (94.9%) students completed the assessments at Time 2,
two years after their rst assessment.
A total of 116 schools had taken the survey up to four consecutive years. Twenty-one of these schools had one or more missing
data points and were excluded from the analyses. The remaining 95 schools were used to assess (relative) long-term changes over a
period of three years (Study 2). The average levels of being bullied or bullying others (2 or 3 times a month or more) at baseline were
only marginally dierent for these schools than for the larger groups of schools they were part of in the full-sample analyses: 21.6%
versus 20.8% for being bullied, and 9.6% versus 10.1% for bullying others. A total of 31,620 students completed baseline assessment
(T0), and 29,814 (94.1%) students in grades 311 completed assessments at Time 3. Demographic information for participants in the
2-year and 3-year studies is presented in Table 1.
Even though the samples in the two sets of analyses were very large by common standards in the eld, it should be noted that the
number of participating students from grades 911 was much smaller than in the lower grades, constituting only one-third or one-
fourth of the average numbers in grades 38. This will, of course, have a clear eect in terms of signicance testing. It is therefore
important to use other indicators of program eects in addition to signicance tests (below). Also, the samples for the non-White
races/ethnic groups were relatively small. In addition, it should be noted that > 12,000 students (17%) of students in the 2-year
study did not respond to the question about race/ethnic background or answered I do not know.We therefore decided to conduct
analyses on this variable only for subgroups consisting of at least 1000 students.
Although not being part of our target population, it is worth mentioning that the 20 schools that were excluded from Study 1
comprised a total of 5980 students (7.8% of the total number) and the ethnicity and gender distributions of these students did not
dier signicantly from those of the target population. In the longitudinal Study 2, which contains a subsample of the participants
from Study 1, the number of students in the 21 excluded schools amounted to 8001 (20.2% of the total number). In both studies, the
levels of the two key bullying variables were somewhat smaller in the excluded schools (by an average of 2.5 percentage points in
Study 1 and 1.6 percentage points in Study 2). Overall, the students in the excluded schools were basically similar to the students in
the target population, except that their levels of being bullied and bullying other students were somewhat lower.
2.2. Measures
In the following sections, we describe the variables used in the analyses for both Study 1 and Study 2 and provide selected
psychometric information about some of them (also see section Approach to analysis about intraclass correlations as reliability
estimates). Participants completed the Olweus Bullying Questionnaire (OBQ), a 40-item anonymous questionnaire that assesses
students' self-reports of bullying others, being bullied, their own actions and reactions when they witness bullying, their attitudes
about bullying, and their perceptions of the eorts of their teachers to counteract bullying (Olweus, 2007a). Most questions ask
students about their experiences during the past couple of months (Olweus, 2007a;Olweus, 2013). The OBQ is recommended for use
in grades 3 and higher (Olweus et al., 2007).
2.2.1. Being bullied
Students were presented with the following detailed denition of bullying:
We say a student is being bullied when another student, or several other students:
say mean and hurtful things, or make fun of him or her, or call him or her mean and hurtful names
completely ignore or exclude him or her from their group of friends or leave him or her out of things on purpose
hit, kick, push, shove around, or lock him or her inside a room
tell lies or spread false rumors about him or her or send mean notes and try to make other students dislike him or her
and do other hurtful things like that.
When we talk about bullying, these things happen more than just once, and it is dicult for the student being bullied to defend
himself or herself. We also call it bullying when a student is teased more than just once in a mean and hurtful way. But we do not
call it bullying when the teasing is done in a friendly and playful way. Also, it is not bullying when two students of about equal
strength or power argue or ght.
(Olweus, 2007a,p.2)
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
60
Students were asked how often they had been bullied at school in the past couple of months. There were ve response options: I
have not been bullied at school in the past couple of months(coded 1); It has only happened once or twice(coded 2), 2 or 3 times a
month(coded 3), About once a week(coded 4), or Several times a week(coded 5). Following this global question, students were
asked about the frequency with which they had experienced nine specic forms of bullying, capturing direct verbal, direct physical,
indirect/relational, and electronic forms of bullying. Each of the nine questions had the same response alternatives as the global
question. Two bullying victimization scores were computed. A dichotomized Being Bullied Global score was calculated based on
students responding that they had been bullied 2 or 3 times a monthor more often (see e.g., Olweus, 2013;Solberg & Olweus,
2003). In addition, a composite Being Bullied scale score was created for each participant by taking the average of all nine items
(α= 0.85 and α= 0.87 for the 2-year and 3-year analyses, respectively).
2.2.2. Bullying others
Students were also asked a global question about the frequency with which they had bullied other students at school in the past
couple of months (Bullying Others Global score), coupled with questions about nine dierent forms of bullying others. As a parallel to
the Being Bullied scale, a composite Bullying Others scale was created (α= 0.88 and 0.89 for the 2-year and 3-year analyses,
respectively).
Since the psychometric characteristics of the Being Bullied and Bullying Others variables have typically been assessed in the same
studies, we report information on their reliability and validity below. At least seven empirical studies from independent researchers
have reported reliabilities for the bullying victimization and perpetration scales in the range of 0.800.90 (see Breivik & Olweus,
2015;Olweus, 2013). With respect to the global questions of being bullied and bullying others, Solberg and Olweus (2003) found
very high correlations at the school level between the global scores and scale scores (r= 0.79 for being bullied and r= 0.77 for
bullying others). There also is good evidence of construct and convergent validity of the global questions of being bullied and bullying
others (Solberg & Olweus, 2003). Both the two global questions and the corresponding scales have shown substantial correlations (in
the 0.350.40 range) with independent peer ratings of corresponding dimensions in a large-scale project with students in grades 310
(n= 19,780; see Olweus, 2013).
2.2.3. Reactions to bullying
Students were asked about their own (re)actions when they witness bullying. One question assessed the propensity of students to
join in bullying of another student (Do you think you could join in bullying a student whom you don't like?), to which students had
the following response options: Denitely no(coded 0), No(coded 1), No, I don't think so(coded 2), I do not know(coded 3),
Yes, maybe(coded 4) or Yes(coded 5). Responses to this item are likely to capture not only students who actually bully other
students but also those who may have a propensity to bully if they observe it around them. In our large Study 2 sample (measured at
baseline, before intervention), the individual-level (Pearson), correlation between this item and the Bullying Others variable was
clearly positive (r= 0.41), as expected, and the form of the association was monotone-increasing (basically linear).
Students were also asked, When you see a student your age being bullied at school, what do you feel or think?Response options
included: That is probably what he or she deserves(coded 0), I do not feel much(coded 1), I feel a bit sorry for him or her(coded 2),
I feel sorry for him or her and want to help(coded 3). This question is intended to measure the extent to which a student feels empathy
for bullied youth. In our analyses based on the same large sample, the correlation of this item with Bullying Others was negative
(r=0.25), as expected (students with bullying tendencies typically score low on empathy variables; e.g., Olweus, 1993;Olweus &
Endresen, 1998), and the association was monotone-decreasing (basically linear).
2.2.4. Perceptions of class teacher's actions
Students were further asked, Overall, how much do you think your class (homeroom) teacher has done to counteract bullying in
the past couple of months?The ve response options ranged from little or nothing(coded 0) to much(coded 4). This question is
designed to measure students' perceptions of the degree to which educators (and in particular the educator with whom the students
interact the most) actively address bullying, and although there is no published evidence of the validity of this item, it has substantial
face validity.
2.2.5. Demographic questions
Finally, students were asked questions about their sex, grade in school, and their race or ethnicity. For this latter question,
students were asked, How do you describe yourself?and were asked to indicate as many of the following categories that applied to
them: American Indian, Black or African American, Arab or Arab American, Hispanic or Latino, Asian American, White, other, or I
don't know.
2.3. Procedure
Training of school staand implementation of the OBPP followed standard practices. The OBPP was implemented with the
support of local certied OBPP trainer-consultants, who provided a 2-day training and monthly in-person or telephone consultation to
members of each school's BPCC throughout implementation of the OBPP. BPCCs, with assistance from certied OBPP trainers-
consultants, provided a full day of training for all staprior to implementing the program. Schools received all necessary OBPP
materials during the rst year of the program. (For more details about program implementation, see Olweus & Limber, 2010b;
Masiello & Schroeder, 2014.)
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
61
To evaluate the OBPP, classroom teachers distributed the Olweus Bullying Questionnaire (OBQ; Olweus, 2007a) in a pencil/paper
scannable, anonymous format to the students in the relevant grades approximately 34 months before the ocial start of the pro-
gram. Schools either began the program in the fall (shortly after start of the school year) or winter (shortly after winter holidays).
Thus, the month in which the OBQ was administered varied among schools. However, dates of survey administration were carefully
recorded so that new measurements with the same questionnaire were made at the same time of the year one, two and three years
after the rst administration. School personnel received a detailed school-level report of the ndings from the questionnaire (Olweus,
2007b) to assist in their planning and internal evaluation of progress/lack of progress. When school personnel submit their scannable
forms to be processed, they are asked to include an information form, upon which they may indicate whether or not they agree to
share their data with Dan Olweus and his fellow researchers. In this study, all agreed to do so. This study was conducted in com-
pliance with the human subjects review board of the rst author's institution.
2.4. Study design
The design for this quasi-experimental study was an extended age cohort design,as developed by Olweus (Olweus, 2005;
Olweus & Limber, 2010b;Shadish, Cook, & Campbell, 2002). In an extended age cohort design, same-aged students from the same
schools are compared across periods in time. For example, when the eect of a program is evaluated after a one-year period, students
in grade 7 at Time 0 (T0, before intervention) will be compared with students in grade 7 from the same school at Time 1 (T1) one year
later. These students were in grade 6 at T0, and at T1 they have been exposed to the program for approximately eight or nine months.
In such a comparison, possible maturational or age-related dierences between the comparison groups are controlled.
When the groups to be compared belong to the same schools, there are good grounds for assuming that a grade cohort diers in
only minor ways from its contiguous cohort. Usually, the majority of the members in the various grade cohorts have been recruited
from the same relatively stable populations and are also likely to have been students in the same schools for several years. The schools
thus serve as their own controls and in this way, the problem with initial dierences between the groups to be compared can be
largely reduced or avoided. In other words, the pretest(T0) values for the individual schools can be considered good answers to the
critical counterfactual question in all evaluation research: How do we obtain reasonable estimates of what the result would have been
if the intervention subjects had not been exposed to the intervention (e.g., Cook, Shadish, & Wong, 2008)? As has been repeatedly
documented, attempts to statistically correct for preexisting initial dierences in common quasi-experimental designs (with none-
quivalent control and intervention groups) are fraught with great diculties (see e.g., Judd & Kenny, 1981;Shadish et al., 2002;
Weisberg, 1979).
Another strength of the extended age cohort design (with more than two consecutive grade cohorts) is that one or more of the
cohorts may serve as a baseline group in one set of comparisons and as an intervention group in another (see Olweus, 2005, and
Olweus & Limber, 2010b, for details.) It is worth noting that the extended age cohort design has later been used by several other
researchers to evaluate bullying prevention programs (Ertesvåg & Vaaland, 2007;Kärnä, Little, Voeten, Poskiparta, Kaljonen, &
Salmivalli, 2011;Kärnä, Little, Voeten, Poskiparta, Alanen, & Salmivalli, 2011;Salmivalli, Kaukiainen, & Voeten, 2005).
2.4.1. History eects
A possible threat to the internal validity of conclusions about program eects in this design is what is usually called history
eects.Such eects may occur due to general time trends or some irrelevant (subject or environmental) factor that may aect or
have aected the intervention group(s) and not the baseline/comparison group(s). Such threats may be dicult to completely rule
out in a single study if little is known about results from studies with the same outcome variables without intervention. The current
study, however, which used consecutive cohorts of roughly similar schools, allowed us to examine such eects by comparing initial
assessments on key measures of interest for adjacent cohorts. Clear dierences in these initial assessments might indicate some form
of history eect that need to be considered in the interpretation of the results. The absence of such dierences, on the other hand,
would make an explanation or partial explanation of registered changes over time as history eectsconsiderably less likely.
2.4.2. Length of program exposure
In the current project, >95% of the schools took the baseline measurement, T0, in MayJune, and had their program start in the
beginning of the next school year. With evaluation of program eects one year after baseline measurement, students in all grade
cohorts had been exposed to the program for approximately eight-nine months. This is a common situation which has been described
in some detail elsewhere (Olweus, 2005).
When the evaluation of program eects is made two years after the baseline measurement (as is the case for the full sample in the
current project), the exposure for the youngest grade cohort is somewhat dierent than for the older cohorts. In this case, it is natural
to make a distinction between exposure for the relevant grade cohort and exposure for the school to which the grade cohort belongs.
To illustrate this distinction, we can use the example of a typical middle school with grades 6 through 8, and with a feeder school or
feeder schools with lower grades that have not used the OBPP in previous years. In such a situation, students in the youngest (grade 6)
cohort at T0 (C6) are compared with students who were in grade 4 at T0 (C4 at T0) and who were enrolled in grade 6 in the middle
school in August in 2009. At T2 in MayJune 2010, these (intervention) students had thus been directly exposed to (been potentially
inuenced by) the program for about 9 months, from August 2009, to MayJune 2010. However, the students in the next higher
comparison/intervention cohort (C5 at T0) were enrolled in grade 6 in the middle school in August already in 2008. These (inter-
vention) students had thus been directly exposed to the OBPP for approximately 18 months at T2, in MayJune 2010, when they were
compared with the students in grade 7 at T0 (C7), before intervention. Also the comparison/intervention cohort (C6 at T0) for the
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
62
oldest cohort in the school, the grade 8 cohort (C8 at T0) had been directly exposed to the program for approximately 18 months at
T2.
Although this illustration shows that the length of direct exposure to the program is less for the youngest comparison/intervention
group of students than for older grade cohorts, it is important to realize that the length of program exposure to the school also had
been about 18 months for this grade cohort. When the youngest comparison/intervention students (C4 at T0) are enrolled in the
middle school in August 2009, some nine months before evaluation measurement at T2 (in MayJune 2010), they enter a school
where the OBPP was implemented nine months earlier. This is likely to be a considerable advantage for the newcomers since the
school has already worked actively with the program for nine months. Accordingly, it is not unreasonable to expect that program
eects for the youngest cohort will be approximately the same as for older cohorts with a longer direct exposure time. We do not
expect that the dierence in length of direct program exposure will have measurable consequences for the youngest grade cohorts in
the participating schools. Although a detailed test of this assumption would be quite challenging (since there are numerous dierent
grade combinations represented in the sample of schools), one can get a good impression of its credibility by examining the 6th grade
cohort (the youngest grade cohort in a large proportion of the grade 68 schools). When the results of this cohort were compared with
the results of adjacent cohorts, their similarity was striking (see percentage change values and regression coecients in Tables 2 and
3below).
Table 2
Changes in Being Bullied and in Bullying Others (dichotomized global questions) between baseline (T0) and time 2 (T2, two years later): analyses by
individual grade.
Grade N
N0/N2
BT0% T2% AC%
Being bullied global score
3 16,653
8590/8063
0.179
⁎⁎⁎
26.19 22.88 3.31
4 16,856
8546/8310
0.168
⁎⁎
24.90 21.89 3.01
5 17,630
9089/8541
0.205
⁎⁎⁎
25.29 21.62 3.67
6 22,302
11,270/11032
0.207
⁎⁎⁎
21.90 18.57 3.33
7 23,257
11,859/11389
0.220
⁎⁎⁎
19.51 16.28 3.23
8 22,801
11,772/11029
0.058 16.93 16.14 0.79
9 6411
3372/3039
0.198
17.51 14.83 2.68
10 5481
2823/2658
0.160 14.05 12.23 1.82
11 5819
3041/2778
0.134 12.83 11.40 1.43
Bullying others global score
3 16,386
8467/7919
0.434
⁎⁎⁎
5.37 3.55 1.82
4 16,639
8437/8202
0.338
⁎⁎
5.71 4.14 1.57
5 17,497
9023/8474
0.492
⁎⁎⁎
8.39 5.30 3.09
6 22,191
11,222/10969
0.487
⁎⁎⁎
9.92 6.34 3.58
7 23,163
11,826/11337
0.368
⁎⁎⁎
10.97 7.86 3.11
8 22,758
11,762/10996
0.349
⁎⁎⁎
13.44 9.87 3.57
9 6375
3363/3012
0.454
⁎⁎⁎
14.54 9.75 4.79
10 5455
2811/2644
0.438
⁎⁎
13.43 9.10 4.33
11 5799
3022/2777
0.382
⁎⁎
12.85 9.15 3.70
B = unstandardized regression coecient.
AC = absolute change (computed as T0-T2).
p< 0.05.
⁎⁎
p< 0.01.
⁎⁎⁎
p< 0.001.
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
63
2.5. Approach to analysis
The Mplus 7.0 program (Muthén & Muthén, 2012) was used to analyze the multilevel (two-level) data consisting of individuals
nested within schools and program eects based on school-aggregated outcome variables. For the key global and scaled outcome
variables, being bullied and bullying others, the within-school intraclass correlations (the unitreliabilities) varied between 0.021
and 0.041. The aggregate reliabilities for the school-aggregated versions of these variables (based on an average schoolsize of 339
students) were very high, in the 0.870.94 range (Kallestad, Olweus, & Alsaker, 1998,Table 5;Snijders & Bosker, 2012, p. 26).
Accordingly, the schools could be dierentiated on these variables with excellent reliability also when based on single items (e.g., the
global item measuring percent bullied students in a school at a certain time point). The aggregate reliability estimates are of par-
ticular importance because all program eects in the study are based on school-aggregated variables. For the other outcome vari-
ables, the within-school intraclass correlations varied between 0.086 and 0.133; the highest value was obtained for the variable
measuring how much the primary teacher had done to counteract bullying. For all of these variables, the aggregate reliabilities were
in the 0.90's. As evident from the low (but highly signicant) intraclass correlations (percent between school variances), most of the
variance was linked to individual dierences, ranging between 87 and 98%.
Our general model can be described as a Multi-site Block Design (Spybrook, Bloom, Congdon, Hill, Martinez, & Raudenbush
(2011, Chapter 5), where schools represent the blocks or sites. As evident from the description of the study design noted above, same-
aged students (in the same grade) from the same schools are compared across periods in time. This blocking is likely to considerably
increase the power of the analyses.
The general (combined) model is:
=+ +++
Y
yyXuue
ij 00 10 ij 0j 1j ij
where Y
ij
is the outcome variable for an individual student in school j, y
00
is the average school (grand) mean, X
ij
is the treatment
indicator (coded with sets of dummy variables reecting program year), y
10
is the main eect of the treatment (the average dierence
between the treatment conditions), u
0j
the random error associated with the level-2 means, u
1j
the random error associated with the
treatment eects, and e
ij
the random error associated with the students at school j. Program year (T0 = baseline, T1 = one year after
program start, T2 = two years after program start, etc.) is treated as a within-school (grade or grade grouping) treatment indicator
(X
ij
). In this model, we have not specied treatment as a xed factor, but we will not explore further the variance of the treatment
eects (across schools) in this article.
The outcome variables were treated as continuous, with the exception of the two global questions dichotomized into being
bullied/bullying others 2-3 times a month or more(coded 1) versus less than 23 times a month(coded 0). For the latter
variables, results were analyzed with multilevel logistic regression using a logit link (Heck & Thomas, 2015). The MLR estimator was
used in all multilevel analyses except for analyses of possible interaction eects between program and sex with categorical outcome
variables. In these cases, the Bayes estimator was used because it is less computationally demanding than MLR (Muthén &
Asparouhov, 2012). When a signicant program x sex interaction eect was found, analyses were rerun on boys and girls separately
using the MLR estimator.
In order to facilitate identication of possible main developmental trends in the longitudinal analyses (Study 2), we combined the
students into sets of three grades levels, which roughly correspond to elementary, middle, and high school grades (grades 35, 68,
and 911. Doing so provides more stable and replicable estimates than analysis of nine individual grades.
With continuous variables such as the Being Bullied/Bullying Others scales, use of a common individual-level eect size such as
Cohen's dgives results that are misleadingly low, since a majority of the participants have scores of zero and cannot improve. Also,
because program eects are based on school-aggregated variables (Spybrook et al., 2011, Chapter 9), for a number of analyses, we
Table 3
Changes in Being Bullied Scale and Bullying Others scale between baseline (T0) and time 2 (T2): Analyses by individual grade.
Being bullied scale Bullying others scale
Total NB Cohen's d(S) Total NB Cohen's d(S)
Grade 3 16,589 0.099
⁎⁎
0.81 16,501 0.039 0.31
Grade 4 16,960 0.098
⁎⁎
0.86 16,758 0.047
⁎⁎⁎
0.61
Grade 5 17,747 0.107
⁎⁎⁎
1.02 17,576 0.063
⁎⁎⁎
1.00
Grade 6 22,541 0.099
⁎⁎⁎
1.18 22,296 0.079
⁎⁎⁎
0.94
Grade 7 23,488 0.094
⁎⁎⁎
1.21 23,274 0.060
⁎⁎⁎
0.85
Grade 8 23,001 0.040
0.40 22,852 0.059
⁎⁎⁎
0.76
Grade 9 6468 0.120
⁎⁎⁎
1.90 6390 0.056
⁎⁎
1.25
Grade 10 5511 0.084
0.94 5473 0.103
⁎⁎
1.33
Grade 11 5858 0.028 0.44 5816 0.066
1.20
B = unstandardized regression coecients.
Cohen's d (S) = school-level eects.
p< 0.05.
⁎⁎
p< 0.01.
⁎⁎⁎
p< 0.001.
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
64
have calculated school-level Cohen's d's with the between-school standard deviation in the denominator (Hedges, 2007, 2011) rather
the individual-level variant.
A key indicator of program eects for the dichotomized global questions is Absolute Change, AC% (% bullied students at T0 - %
bullied students at T2). This measure has the advantage of being largely independent of the levels of baseline values of the groups
compared and is easy to interpret (Wilkinson, 1999). In our study, it can be directly translated into an estimate of the number of
students who have escaped being bullied (or the number of new victims) after implementation of the program. To facilitate com-
parison with some other studies, we have also reported Relative Change dened as [(% bullied students at T0 - % bullied students at
T2)/% bullied students at T0].
3. Results
Results are presented in two sections. The rst section reports analyses of possible program eects with respect to being bullied
and bullying others for the full sample of students over two years (Study 1). Detailed analyses by grade and grade-groupings are
provided, in addition to analyses by sex and race/ethnicity. The second section provides more detailed examinations of longer-term
eects (over three years) for the sub-group of 95 schools for whom such data were available (Study 2). In all these analyses, schools
belonging to dierent school cohorts (2008, 2009, and 2010 cohorts) were aligned. To achieve more stable estimates, most analyses
were based on grade groupings (grades 35, 68, and 911, which corresponds to elementary, middle, and high school grades in most
schools) rather than individual grades.
3.1. Study 1: analyses for the full sample over two years
3.1.1. Grade-level analyses for being bullied
Table 2 and Fig. 1 present relevant data for the dichotomized Being Bullied Global variable. All grades showed reductions, and the
changes over time were signicant for all but three grades (8th, 10th, and 11th grades). Average Absolute Change (unweighted)
amounted to 2.58% and varied between 0.79% (8th grade) and 3.67% (5
th
grade). Corresponding analyses for the scale scores of being
bullied are presented in Table 3. Signicant changes from T0 to T2 were observed for all but one grade (11th). With the exception of
grades 8 and 11, all school level eect sizes [Cohen's d(S)] were large to very large. Average change scores (Table 2) and the
regression coecient for grade 11 (Table 3) indicated that the program eects for being bullied seemed overall to be somewhat
weaker in the high school grades than in elementary and middle school grades.
3.1.2. Grade-level analyses for bullying others
Similar analyses were conducted for the dichotomized Bullying Others Global variable (Table 2) and for the related scale scores
(Table 3). Signicant reductions in Bullying Others Global were observed for all grades. Average Absolute Change (unweighted)
amounted to 3.28% and varied between 1.57% (4th grade) and 4.79% (9th grade). With regard to students' scaled scores of bullying
others, signicant changes from T0 to T2 were observed for all grades except 3rd grade. Again, school level eect sizes were large to
very large with a few exceptions (3rd and 4th grades). In terms of relative strength of program eects at dierent grade levels,
Fig. 1. Change in Being Bullied Global after implementation of the OBPP (2-year study, 210 schools). Analyses by grade.
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
65
average change scores were actually higher in high schools grades than other grades.
3.1.3. Examination of eects for sex and race
Analyses of the two dichotomized global variables are presented in Table 4. For ve out of six analyses, the same regression
coecient could describe the signicant positive program eects for boys and girls. Signicant positive decreases over time were
observed for both dichotomous variables for all three grade levels. In addition, a program by sex interaction was observed for
students in grades 35 for the Being Bullied Global score (unstandardized beta = 0.099, p< 0.05, CI (90%) = 0.0460.140), in-
dicating somewhat greater reductions for boys than girls.
To examine the extent to which dierential program eects were found for students of dierent races/ethnic groups, we con-
ducted separate analyses on the two dichotomous global variables for students who identied themselves as Black, Hispanic, and
White (Table 5). As mentioned, analyses were reported only if at least a total of 1000 students were represented within a group. As a
Table 4
Changes in Being Bullied and Bullying Others (dichotomized global questions) between baseline (T0) and time 2 (T2): Analyses by sex and grade
groupings.
Grades Total NB T0% T2% AC (%)
Being bullied global score
35 Girls: 25,188
Boys: 25,838
Girls: 0.115
⁎⁎
Boys: 0.274
⁎⁎⁎
Girls: 26.43
Boys: 24.75
Girls: 24.26
Boys: 20.00
Girls: 2.17
Boys: 4.75
68 68,360 0.161
⁎⁎⁎
19.93 17.48 2.45
911 17,711 0.136
14.91 13.27 1.64
Bullying others global score
35 50,522 0.449
⁎⁎⁎
6.73 4.40 2.33
68 68,112 0.389
⁎⁎⁎
11.18 7.85 3.32
911 17,629 0.414
⁎⁎⁎
13.80 9.57 4.17
B = unstandardized regression coecient.
AC = absolute change (computed as T0-T2).
p< 0.05.
⁎⁎
p< 0.01.
⁎⁎⁎
p< 0.001.
Table 5
Changes in Being Bullied and Bullying Others (dichotomized Global Questions) between baseline (T0) and time 2 (T2): Analyses by race/ethnicity
and grade groupings.
Grades B(program eects) Baseline (T0)% T2% AC (%) Total N
Black students: being bullied global score
350.165 23.90 21.03 2.87 2177
680.041 14.70 14.19 0.51 3616
Hispanic students: being bullied global score
350.158 25.42 22.53 2.89 1462
680.092 14.18 13.09 1.09 4595
White students: being bullied global score
350.191
⁎⁎⁎
25.07 21.66 3.41 28,849
680.190
⁎⁎⁎
19.98 17.12 2.86 42,529
911 0.204
⁎⁎
13.78 11.52 2.26 12,644
Black students: bullying others global score
350.129 12.94 11.55 1.39 2155
680.347
⁎⁎
17.88 13.34 4.54 3579
Hispanic students: bullying others global score
350.356 8.88 6.39 2.49 1454
680.217(
)14.09 11.66 2.43 4592
White students: bullying others global score
350.444
⁎⁎⁎
6.00 3.93 2.07 28,571
680.482
⁎⁎⁎
10.33 6.64 3.69 42,544
911 0.507
⁎⁎⁎
10.97 6.91 4.06 12,656
B = unstandardized regression coecient.
AC = absolute change (computed as T0 T2).
p< 0.05.
⁎⁎
p< 0.01.
⁎⁎⁎
p< 0.001.
()
p= 0.052.
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
66
consequence, we did not analyze the responses from Hispanic and Black students in the 911 grade grouping. Program eects were
typically somewhat larger for White students but also signicant for Black middle school students (bullying others) and Hispanic
middle school students (bullying others).
3.1.4. Analyses of possible history eects
To examine historical eects, we used multiple group analyses and compared initial assessments (at baseline) on the two key
measures of interest for the three cohorts beginning with the program in 2008, 2009, and 2010, respectively. Satorra Bentler Chi-
square dierence test (TRd) analyses revealed no signicant dierences across the cohorts with regard to the percentage of students
being bullied (global question; TRd = 0.99, df =2, p> 0.05), or the percentage of students bullying others (global question;
TRd = 3.42, df =2,p> 0.05), suggesting that the changes over time were not due to historical eects.
3.2. Study 2: analyses for the longitudinal sample over three years
Detailed analyses were conducted to assess somewhat longer-term and year-by-year changes in students' responses on variables of
interest. In these analyses, we used grade groupings (35, 68, 911).
3.2.1. Students' reports of being bullied and bullying others
Table 6 presents slope values (program eects) for students' reports of being bullied and bullying others, based on the scale scores
(see also Fig. 2). Among students in grades 35, there were signicant reductions in students' reports of both variables between
baseline (T0) and T1, between baseline and T2, and between baseline and T3. Moreover, there were reductions between T1 and T2
and T1 and T3 (far right column). Similar, though somewhat less marked results were obtained for the 68 grouping. For students in
the 911 grade grouping, signicant change occurred only after three years. By and large, eects became gradually stronger over time
and were greatest after three years.
3.2.2. Students' reactions to bullying
Changes over time were analyzed for students' reports of how they feel when they see a student their age being bullied (upper
panel of Table 7). Generally, there were increases over time in expressions of empathy for a bullied peer in all grade groupings. For
students in grades 912, however, clear and signicant increases did not emerge until T3. School-level eect sizes varied between
moderate and large. Table 7 also presents changes over time in students' willingness to join in bullying a peer whom they don't like.
Steady decreases over time were observed in students' reports for all grade groupings. The changes from one assessment to the next
were marked and the strongest eects were seen after three years (TO vs T3). The school-level eects sizes were large overall.
3.2.3. Perceptions of class teacher's actions to address bullying
Table 7 also presents changes over time in students' assessment of how much their class or homeroom teacher had done to
counteract bullying. In all grade groupings, signicant and marked increases were observed in students' perceptions that their
Table 6
Changes in Being Bullied Scale and Bullying Others scale across four time periods.
Grades Total NB
T0 vs. T1
B
T0 vs. T2
B
T0 vs. T3
d(S)
T0 vs. T3
Additional contrasts
Being bullied scale score
35 51,669 0.075
⁎⁎⁎
0.111
⁎⁎⁎
0.120
⁎⁎⁎
d= 1.05 T1 vs. T2
⁎⁎
T1 vs. T3
⁎⁎
68 49,964 0.056
⁎⁎⁎
0.072
⁎⁎⁎
0.079
⁎⁎⁎
d= 0.94 T1 vs. T3
911 20,433 0.032 0.039 0.076
⁎⁎
d= 1.20 T1 vs. T3
T2 vs. T3
Bullying others scale score
35 50,996 0.030
⁎⁎⁎
0.048
⁎⁎⁎
0.058 d= 0.75 T1 vs. T2
⁎⁎
T1 vs. T3
⁎⁎⁎
T2 vs. T3
68 49,522 0.037
⁎⁎⁎
0.067
⁎⁎⁎
0.092
⁎⁎⁎
d= 1.30 T1 vs. T2
⁎⁎⁎
T1 vs. T3
⁎⁎⁎
T2 vs. T3
⁎⁎⁎
911 20,249 0.029 0.052 0.090
⁎⁎
d= 1.27 T1 vs. T3
⁎⁎
T2 vs. T3
T0 = baseline, T1 = Time 1 (one year later), T2 = Time 2 (two years later), T3 = Time 3 (three years later); B= unstandardized regression
coecient; d (S) = school-level eects.
p< 0.05.
⁎⁎
p< 0.01.
⁎⁎⁎
p< 0.001.
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
67
primary teacher had increased his or her eorts to address bullying. However, the general lack of signicant changes between the
various time periods after baseline (T1 vs. T2, T2 vs. T3), in particular for the two highest groupings, shows that there had occurred a
change between baseline and T1 which was maintained thereafter but not increased further. School-level eects sizes were large, on
average.
Fig. 2. Change in Bullying others scale scores after implementation of OBPP (3-year study, 95 schools). Analyses by grade groupings (grades 35,
68, and 911).
Table 7
Changes in students' perceptions and actions across four time periods.
Grades Total NB
T0 vs. T1
B
T0 vs. T2
B
T0 vs. T3
d(S) Additional contrasts
Students' perceptions of how they feel when a student is bullied
35 51,056 0.007 0.030
0.063
⁎⁎⁎
d= 0.75 T1 vs. T3
⁎⁎⁎
T2 vs. T3
⁎⁎
68 49,450 0.054 0.030
0.079
⁎⁎⁎
d= 0.51 T1 vs. T2
⁎⁎⁎
T1 vs. T3
⁎⁎⁎
T2 vs. T3
⁎⁎
911 20,135 0.010 0.000 0.090
d= 0.79 T1 vs. T3
⁎⁎
T2 vs. T3
⁎⁎⁎
Students' perceptions of whether they think they could join in bullying
35 51,374 0.104
⁎⁎⁎
0.118
⁎⁎⁎
0.234
⁎⁎⁎
d= 1.02 T1 vs. T3
⁎⁎⁎
T2 vs. T3
⁎⁎⁎
68 49,575 0.024 0.197
⁎⁎⁎
0.300
⁎⁎⁎
d= 0.99 T1 vs. T2
⁎⁎⁎
T1 vs. T3
⁎⁎⁎
T2 vs. T3
⁎⁎⁎
911 20,211 0.027 0.151
⁎⁎
0.265
⁎⁎⁎
d= 1.25 T1 vs. T2
T1 vs. T3
⁎⁎⁎
T2 vs. T3
Students' perceptions that their teacher had addressed bullying
35 51,046 0.265
⁎⁎⁎
0.353
⁎⁎⁎
0.323
⁎⁎⁎
d= 1.38 T1 vs. T2
⁎⁎
T1 vs. T3
68 49,438 0.261
⁎⁎⁎
0.302
⁎⁎⁎
0.259
⁎⁎⁎
d= 0.58
911 20,124 0.273
⁎⁎⁎
0.314
⁎⁎⁎
0.334
⁎⁎⁎
d= 2.56
T0 = baseline, T1 = Time 1 (one year later), T2 = Time 2 (two years later), T3 = Time 3 (three years later); B= unstandardized regression
coecient; d (S) = school-level eects.
p< 0.05.
⁎⁎
p< 0.01.
⁎⁎⁎
p< 0.001.
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
68
4. Discussion
The purpose of this study was to evaluate a large-scale implementation of the Olweus Bullying Prevention Program with children
and youth in grades 311 in the United States. Given increasing concerns about the markedly negative short- and long-term eects of
being bullied on individuals who are targeted (Olweus, 2013;Ttoet al., 2011a) and evidence that regular engagement in bullying
other students is related to later antisocial and criminal behavior (Olweus, 1993, 2011;Ttoet al., 2012), the prevention of bullying
is of utmost importance. In light of the proliferation of prevention and intervention programs and questions about the eectiveness of
such eorts, there is a need for systematic evaluations of commonly-used prevention programs to reduce bullying.
The results of this large-scale study provide support for the eectiveness of the Olweus Bullying Prevention Program in the
participating schools. Among students in all grades, there were clear reductions in the two key dimensions, being bullied and bullying
other students, and results were generally consistent for both global questions and scaled scores. In most analyses, program eects
were of similar magnitude for boys and girls but somewhat weaker and less comprehensive for students who identied themselves as
Black or Hispanic, compared to the White majority students.
The average Absolute Change score over the two-year period amounted to somewhat < 3%. This change score can be translated
into a rough estimate of nearly 2000 students who have escaped being bullied in the two-year period (and very likely, longer) and a
similar number who have stopped bullying other students. Examination of year-to-year changes among the longitudinal sample of
participants over three years conrmed and extended the reported ndings. In particular, these analyses documented overall in-
creases in students' expressions of empathy with bullied peers, marked decreases in their willingness to join in bullying, and clear
perceptions that their primary teacher had increased his or her eorts to address bullying. The changes with regard to empathy and
(un)willingness to join in bullying strongly suggest that the program had helped shift student attitudes to bullying and related
behaviors to a more positive and inclusive school climate. In addition, the longitudinal analyses showed that program eects were
generally larger, the longer the program had been in place.
Our ndings are relevant for both educators and researchers. They provide strong empirical support for the position that a
research based, whole-school bullying prevention program like the OBPP is likely to have systematic positive eects among U.S.
students, in contrast to general concerns about the likely ineectiveness of programs developed outside of the U.S (Cohen et al., 2015;
Evans et al., 2014;Hong & Espelage, 2012). In addition, they provide educators, parents and policymakers guidance about eective
strategies to reduce bullying in schools.
The positive program eects were generally consistent with our expectations and the ndings of several evaluations of the OBPP
in Norway (e.g., Olweus & Limber, 2010a, 2010b) but were more consistently positive than previous studies of the OBPP in the U.S.
(Bauer et al., 2007;Limber et al., 2004). The reductions in bullying victimization and perpetration are also consistent with the
general message from the meta-analysis of Ttoand Farrington (2009, 2011), that research-based bullying prevention programs can
be eective.
With regard to the issue of the relative strength of program eects at dierent age/grade levels, the eects on being bullied were
weaker in higher grades. To illustrate, in Study 1, the (unweighted) average Absolute Change score for grades 37was 3.31% and
1.68% for the higher grades, 811. We also found no signicant program eects among 11th graders. Moreover, in the 3-year
longitudinal analyses of Study 2 (Tables 6 and 7), eects were generally somewhat weaker and took longer time to achieve in the
highest grade grouping comprising grades 911. This was in line with our predictions, and may be explained, in part, by contextual
dierences in the school environments of high schools vs. elementary or middle schools (e.g., size of the student body and sta,
multiple teachers for each student and somewhat dierent denitions of the teacher's role, less exible schedules) that may aect
students' sense of community and ease of implementation of the program (Limber, 2010;Olweus & Kallestad, 2010). The fact that we
observed less positive results with regard to being bullied for the 8th grade students was unexpected and warrants further in-
vestigation.
For bullying others, results were somewhat inconsistent. The average Absolute Change score for the lower grades (grades 37,
Table 2) was 2.63% whereas the score for grades 811 was 4.10%, actually suggesting stronger program eects in higher grades.
However, as mentioned in the section on being bullied, the 3-year longitudinal analyses (Tables 6 and 7) generally suggested
somewhat weaker eects in the 911 grade grouping than in the lower grades at Time 3. These longitudinal results are rather
consistent with results from Norwegian evaluations, where program eects at higher grade levels have generally been somewhat
weaker and have taken longer to obtain (Olweus & Limber, 2010b). Since dierent indicators pointed in opposite directions, the
empirical results do not permit a strong and clear conclusion on this issue for the bullying others variables in this sample. Future
research is warranted to further clarify possible grade-level dierences in program eects and explore possible rationales.
For the most part, our ndings were consistent, whether based on dichotomous (global) variables or scale scores of being bullied
or bullying others. Of the 36 analyses that examined changes in being bullied and bullying others across 9 grades, consistent ndings
using dichotomous and scale scores were observed in all but 5 cases. With respect to students' reports of being bullied, analyses using
the dichotomous variable revealed no signicant program eects among students in grades 8, 10, or 11, whereas analyses using the
scale score revealed signicant program eects for all grades, with the exception of 11th. However, an analysis of the eect sizes of
the scale scores for the 8th and 10th grade students (see Table 3) revealed they were somewhat weaker than most other grades. There
also was one discrepancy in ndings between the dichotomous variable for bullying others and the corresponding scaled score.
Although signicant program eects were observed for all age groups using the dichotomous variable, no signicant program eects
were observed for 3rd graders when using the scale score.
Consistent with expectations (Olweus & Limber, 2010b) and evaluations of other prevention programs outside of the U.S. that
address bullying (Kärnä, Little, Voeten, Poskiparta, Kaljonen, & Salmivalli, 2011;Kärnä, Little, Voeten, Poskiparta, Alanen, &
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
69
Salmivalli, 2011), we observed signicant reductions in being bullied and bullying others for both boys and girls. Although the
majority of analyses indicated similar program eects for boys and girls, there was also a signicant program by sex interaction in the
grade 35 grouping in favor of boys. Although these results might suggest that the program is somewhat more eective in elementary
grades among boys than girls, the dierence should be interpreted cautiously, as there were no other program by sex interactions. The
issue of possible dierential program eects for boys and girls will benet from more detailed analyses of dierent forms of bullying
(such as physical and relational bullying, for example), as research has documented gender dierences in the frequency of some forms
of bullying (Zhang et al., 2016)
We tentatively examined program eects for White, Black, and Hispanic students and observed signicant reductions in in-
volvement in bullying for 8 of 14 possible analyses (Table 5). Although some of the percentage reductions among Black and Hispanic
students were of about the same size as the reductions among White students, results did not reach signicance for the ethnic
minorities due to considerably smaller numbers of responding students. Nonetheless, the overall weaker program eects for Black
and Hispanic students deserves careful attention. Additional research is needed to understand how students of dierent races and
ethnicities understand, experience, and engage in bullying (Wang, 2013) and the extent to which such students (and possibly adults
within their schools) are receptive or non-receptive to particular prevention and intervention strategies.
Students of all ages were generally positive in their assessments of their own teachers' actions to address bullying. These positive
changes appeared after one year for all grade levels and the higher levels of teacher engagement were maintained over time. The fact
that the teachers (according to their students) maintained the increased levels of their eorts over three years must be viewed as a
very good sign, and in general agreement with the OBPP goal of making key elements of the program a natural, permanent part of the
schools' everyday life and culture. More generally, these and other reported results can also be seen as conrming evidence that the
program had achieved positive changes with regard to bullying in these schools and that the changes had been marked enough for the
students to perceive them.
4.1. Strengths and limitations
One clear strength of this study was the large sample size and the wide range of grade levels included. Although the numbers of
schools involved in evaluations of bullying prevention programs seldom surpass 20 schools (Farrington & Tto, 2009), our research
followed students from 210 schools over two years (Study 1) and students from the sub-sample of 95 schools over three years (Study
2). Moreover, whereas many evaluations have focused on more narrow age ranges (e.g., elementary grades [Waasdorp, Bradshaw, &
Leaf, 2012] or middle school grades [Espelage, Low, Polanin, & Brown, 2013]), this study assessed students across nine grades.
However, our ndings can be primarily generalized to schools in Pennsylvania with a demographic predominance of White students
from schools in primarily suburban and rural locales and, only by reasonable assumption, to similar schools in other states in the U.S.
Outcome variables were limited to student self-reports, assessing both bullying perpetration and bullying victimization through
global questions and scaled scores. As mentioned, these questions and scales have been shown to have a number of good psycho-
metric qualities. In addition, it is important to realize that all program eects in the current study were based on school-aggregated
outcome variables which are much more reliable (typically, in the high 0.80's or 0.90's) than corresponding (single) items at the
individual level. Moreover, we assessed changes in students' perceptions of their own attitudes and reactions as potential witnesses to
bullying. The positive changes on these supplementaryvariables were as expected and lend additional support to the results on the
key outcome variables, being bullied and bullying others.
Although the results of the current evaluations were generally consistent with the ndings of a number of evaluations of the OBPP
in Norway (Olweus, 2005;Olweus & Limber, 2010b), it should also be noted that program eects were somewhat weaker/took
somewhat longer time to achieve in the U.S. Additional research is needed to examine what factors might explain such a dierence if
it is replicated. This study did not analyze delity of implementation of the OBPP. In future studies of this and other bullying
prevention programs, it will be important to examine the extent to which program implementation is related to outcomes.
This study did not employ an experimental design using random assignment of schools to intervention or control conditions.
Although randomized controlled trials (RCTs) are usually, and often somewhat uncritically, considered the gold standardin all
evaluation research, there are often serious problems realizing such a design in practice, particularly with large, complex organi-
zations such as schools (Olweus, 2005;Shadish et al., 2002;Weisberg, 1979). In many cases, a good quasi-experimental design, such
as the extended age cohort design used in the current study, can be considered a strong alternative for the study of program eects.
The extended age cohort design used in the present evaluation can actually be seen as an example of intact group matchingwhich is
one of the (few) conditions under which observational/quasi-experimental studies are likely to produce causal estimates that are
comparable to those obtained with experimental (randomized controlled) studies (Cook et al., 2008).
Another challenge involves ruling out possible eects of other school-based programs that may have been implemented within
intervention schools during the evaluation period. For several reasons, such eects are not likely to explain our results. First, as
mentioned in the introduction, there are very few, if any, programs that have documented systematic bullying-reducing eects in the
U.S. Even if there existed such a program, it is highly unlikely that it had used a temporal pattern of implementation that was largely
aligned with our design. In addition, by virtue of participating in this study, school administrators made commitments not to im-
plement other bullying or violence prevention programs during the intervention period.
In summary, this study provides strong support for the eectiveness of the OBPP among U.S. students in elementary, middle, and
early high school grades. Although our results cannot be widely generalized beyond schools with similar demographics to those in the
current study, the ndings provide clear evidence that a program developed in Norway can (with some adaptations to cultural and
educational conditions) also be eective in reducing bullying among students in the U.S. Although an absolute reduction by 24%
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
70
may supercially not look very impressive, it means in practice that a considerable number of children and youth in the project have
escaped bullying, have experienced an improved school situation, and, in many cases, have been provided a platform for a positive
turning point in their lives. Future research building on the experiences from the current large-scale project and using data on delity
of program implementation will likely contribute to making the program even more eective in the United States.
Acknowledgements
This research was supported by generous funding from the Highmark Foundation. The authors wish to thank The Highmark
Foundation, colleagues at the Center for Health Promotion and Disease Prevention, and colleagues at the Center for Schools and
Communities for their remarkable support of the intervention program and evaluation.
References
Bauer, N. S., Lozano, P., & Rivara, F. P. (2007). The eectiveness of the Olweus Bullying Prevention Program in public middle schools: A controlled trial. Journal of
Adolescent Health, 40, 266274. http://dx.doi.org/10.1016/j.jadohealth.2006.10.005.
Berger, K. S. (2007). Update on bullying at school: Science forgotten? Developmental Review, 27,90126. http://dx.doi.org/10.1016/j.dr.2006.08.002.
Breivik, K., & Olweus, D. (2015). An item response theory analysis of the Olweus Bullying scale. Aggressive Behavior, 41,113. http://dx.doi.org/10.1002/ab.21571.
Cohen, J., Espelage, D., Twemlow, S., Berkowitz, M. W., & Comer, J. P. (2015). Rethinking eective bully and violence prevention eects: Promoting healthy school
climates, positive youth development, and preventing bully-victim-bystander behavior. Review of Educational Research, 15,240.
Cook, T. D., Shadish, W. R., & Wong, V. C. (2008). Three conditions under which experiments and observational studies produce comparable causal estimates: New
ndings from within-study comparisons. Journal of Policy Analysis and Management, 27, 724750. http://dx.doi.org/10.1002/pam.20375.
Cook, R. C., Williams, K. R., Guerra, N. G., Kim, T. E., & Sadek, S. (2010). Predictors of bullying and victimization in childhood and adolescence: A meta-analytic
investigation. School Psychology Quarterly, 25,6583. http://dx.doi.org/10.1037/a0020149.
Ertesvåg, S. K., & Vaaland, G. S. (2007). Prevention and reduction of behavioural problems in school: An evaluation of the Respect program. Educational Psychology, 27,
713736. http://dx.doi.org/10.1080/01443410701309258.
Espelage, D. L., Low, S., Polanin, J. R., & Brown, E. C. (2013). The impact of a middle school program to reduce aggression, victimization, and sexual violence. Journal
of Adolescent Health, 53, 180186. http://dx.doi.org/10.1016/j.jadohealth.2013.02.021.
Evans, C. B., Fraser, M. W., & Cotter, K. L. (2014). The eectiveness of school-based bullying prevention programs: A systematic review. Aggression and Violent Behavior,
19, 532544. http://dx.doi.org/10.1016/j.avb.2014.07.004.
Farrington, D. P., & Tto, M. M. (2009). School-based programs to reduce bullying and victimization: A systematic review. Campbell Systematic Reviews, 6,1149.
Gladden, R. M., Vivolo-Kantor, A. M., Hamburger, M. E., & Lumpkin, C. D. (2014). Bullying surveillance among youths: Uniform denitions for public health and
recommended data elements, version 1.0. Atlanta. GA: National Center for Injury Prevention and Control. Centers for Disease Control and Prevention and U.S.
Department of Education.
Heck, R. H., & Thomas, S. L. (2015). An introduction to multilevel modelling techniques: MLM and SEM approaches using Mplus. New York: Routledge.
Hedges, L. (2007). Eect sizes in cluster-randomized designs. Journal of Educational and Behavioral Statistics, 32, 341370. http://dx.doi.org/10.3102/
1076998606298.
Hedges, L. (2011). Eect sizes in three-level cluster-randomized experiments. Journal of Educational and Behavioral Statistics, 36, 346380. http://dx.doi.org/10.3102/
10769986103766.
Hong, J. S., & Espelage, D. L. (2012). A review of research on bullying and peer victimization in school: An ecological system analysis. Aggression and Violent Behavior,
17, 312322. http://dx.doi.org/10.1016/j.avb.2012.03.003.
Judd, C. M., & Kenny, D. A. (1981). Estimating the eects of social interventions. New York: Cambridge University Press.
Kallestad, J. H., Olweus, D., & Alsaker, F. (1998). School climate reports from Norwegian teachers: A methodological and substantive study. School Eectiveness and
School Improvement, 9,7094. http://dx.doi.org/10.1080/0924345980090104.
Kärnä, A., Voeten, M., Little, T. D., Poskiparta, E., Alanen, E., & Salmivalli, C. (2011a). Going to scale: A nonrandomized nationwide trial of the KiVa antibullying
program for grades 19. Journal of Consulting and Clinical Psychology, 79, 796805. http://dx.doi.org/10.1037/a0025740.
Kärnä, A., Voeten, M., Little, T. D., Poskiparta, E., Kaljonen, A., & Salmivalli, C. (2011b). A large-scale evaluation of the KiVa antibullying program: Grades 46. Child
Development, 82, 311330. http://dx.doi.org/10.1111/j.1467-8624.2010.01557.x.
Limber, S. P. (2010). Implementation of the Olweus Bullying Prevention Program: Lessons learned from the eld. In D. Espelage, & S. Swearer (Eds.). Bullying in North
American schools: A social-ecological perspective on prevention and intervention (pp. 291306). (2nd ed.). New York: Routledge.
Limber, S. P., Nation, M., Tracy, A. J., Melton, G. B., & Flerx, V. (2004). Implementation of the Olweus Bullying Prevention programme in the southeastern United
States. In P. K. Smith, D. Pepler, & K. Rigby (Eds.). Bullying in schools: How successful can interventions be? (pp. 5579). Cambridge, UK: Cambridge Press.
Limber, S. P., & Olweus, D. (2017). Lessons learned from scaling-up the Olweus Bullying Prevention Program. In C. Bradshaw (Ed.). Handbook on bullying prevention: A
life course perspective (pp. 189199). Washington, DC: National Association of Social Workers Press.
Luxenberg, H., Limber, S. P., & Olweus, D. (2015). Bullying in U.S. schools: 2014 status report. Center City, MN: Hazelden Foundation.
Masiello, M. G., & Schroeder, D. (2014). A public health approach to bullying prevention. Washington, DC: American Public Health Association.
Muthén, B. O., & Asparouhov, T. (2012). Bayesian SEM: A more exible representation of substantive theory. Psychological Methods, 17, 313335. http://dx.doi.org/
10.1037/a0026802.
Muthén, L. K., & Muthén, B. O. (2012). Mplus User's Guide (7th Ed). Los Angeles, CA: Muthén and Muthén.
National Academies of Sciences, Engineering, and Medicine (2016). Preventing bullying through science, policy, and practice. Washington, DC: The National Academies
Press.
National Center for Education Statistics (2014). What are locale codes? Retrieved from https://nces.ed.gov/ccd/rural_locales.asp.
Oce of the Special Representative of the Secretary-General on Violence against Children (2016). Ending the torment: Tackling bullying from the schoolyard to cyberspace.
New York: United Nations.
Olweus, D. (1973). Hackkycklingar och översittare. Forskning om skolmobbing. Stockholm: Almqvist & Wicksell.
Olweus, D. (1978). Aggression in the schools: Bullies and whipping boys. Washington, DC: Hemisphere Press.
Olweus, D. (1991). Bully/victim problems among schoolchildren: Basic facts and eects of a school based intervention program. In D. J. Pepler, & K. H. Rubin (Eds.).
The development and treatment of childhood aggression (pp. 411448). Hillsdale, NJ: Erlbaum.
Olweus, D. (1993). Bullying at school: What we know and what we can do. New York: Blackwell.
Olweus, D. (1997). Bully/victim problems in school: Facts and intervention. European Journal of Psychology of Education, 12, 495510. http://dx.doi.org/10.1007/
BF03172807.
Olweus, D. (2005). A useful evaluation design, and eects of the Olweus Bullying Prevention Program. Psychology, Crime & Law, 11, 389402. http://dx.doi.org/10.
1080/10683160500255471.
Olweus, D. (2007a). Olweus Bullying Questionnaire. Center City, MN: Hazelden.
Olweus, D. (2007b). Olweus Bullying Questionnaire: Standard school report. Center City, MN: Hazelden.
Olweus, D. (2011). Bullying at school and later criminality: Findings from three Swedish community samples of males. Criminal Behaviour and Mental Health, 21,
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
71
151156.
Olweus, D. (2013). School bullying: Development and some important challenges. Annual Review of Clinical Psychology, 9, 751780. http://dx.doi.org/10.1146/
annurev-clinpsy-050212-185516.
Olweus, D., & Endresen, I. (1998). The importance of sex-of-stimulus object: Age trends and sex dierences in empathic responsiveness. Social Development, 7,
370388. http://dx.doi.org/10.1111/1467-9507.00073.
Olweus, D., & Kallestad, J. H. (2010). The Olweus Bullying Prevention Program: Eects of classroom components at dierent grade levels. In K. Osterman (Ed.).
Indirect and direct aggression (pp. 113131). New York: Peter Lang.
Olweus, D., & Limber, S. P. (2007). Olweus Bullying Prevention Program: Teacher guide. Center City, MN: Hazelden.
Olweus, D., & Limber, S. P. (2010a). Bullying in school: Evaluation and dissemination of the Olweus Bullying Prevention Program. American Journal of Orthopsychiatry,
80, 124134. http://dx.doi.org/10.1111/j.1939-0025.2010.01015.x.
Olweus, D., & Limber, S. P. (2010b). The Olweus Bullying Prevention Program: Implementation and evaluation over two decades. In S. R. Jimerson, S. M. Swearer, & D.
L. Espelage (Eds.). Handbook of bullying in schools: An international perspective (pp. 377401). New York: Routledge.
Olweus, D., Limber, S. P., Flerx, V., Mullin, N., Riese, J., & Snyder, M. (2007). Olweus Bullying Prevention Program: Schoolwide guide. Center City, MN: Hazelden.
Salmivalli, C., Kaukiainen, A., & Voeten, M. (2005). Intervention: Implementation and outcome. British Journal of Educational Psychology, 75, 465487. http://dx.doi.
org/10.1348/000709905X26011.
Salmivalli, C., Lagerspetz, K., Björkqvist, K., Österman, K., & Kaukiainen, A. (1996). Bullying as a group process: Participant roles and their relations to social status
within the group. Aggressive Behavior, 22,115. http://dx.doi.org/10.1002/(SICI)1098-2337(1996)22:1<1::AID-AB1>3.0.CO;2-T.
Salmivalli, C., & Voeten, M. (2004). Connections between attitudes, group norms, and behaviour in bullying situations. International Journal of Behavioral Development,
28, 246258. http://dx.doi.org/10.1080/01650250344000488.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental design for generalized causal inference. Boston: Houghton-Miin.
Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis (2nd edition). London: Sage.
Solberg, M. E., & Olweus, D. (2003). Prevalence estimation of school bullying with the Olweus Bully/Victim Questionnaire. Aggressive Behavior, 29, 239268. http://
dx.doi.org/10.1002/ab.10047.
Spybrook, J., Bloom, H., Congdon, H., Hill, C., Martinez, A., & Raudenbush, S. (2011). Optimal design plus empirical evidence: Documentation for the Optimal
Designsoftware. Retrieved from: http://hlmsoft.net/od/od-manual-20111016-v300.pdfGoogle Scholar.
Stavrindes, P., Georgiou, S., & Theofanous, V. (2010). Bullying and empathy: A short-term longitudinal investigation. Educational Psychology, 30, 793802. http://dx.
doi.org/10.1080/01443410.2010.506004.
Tto, M. M., Eisner, M., & Bradshaw, C. P. (2014). Bullying prevention: Assessing existing meta-evaluations. Encyclopedia of Criminology and Criminal Justice (pp. 231
242). Springer New York.
Tto, M., & Farrington, D. (2009). What works in preventing bullying: Eective elements of programmes. Journal of Aggression, Conict and Peace Research, 1,1324.
http://dx.doi.org/10.1108/17596599200900003.
Tto, M. M., & Farrington, D. P. (2011). Eectiveness of school-based programs to reduce bullying: A systematic and meta-analytic review. Journal of Experimental
Criminology, 7,2756. http://dx.doi.org/10.1007/s11292-010-9109-1.
Tto, M. M., Farrington, D. P., & Lösel, F. (2012). School bullying as a predictor of violence later in life: A systematic review and meta-analysis of prospective
longitudinal studies. Aggression and Violent Behavior, 17, 405418. http://dx.doi.org/10.1016/j.avb.2012.05.002.
Tto, M. M., Farrington, D. P., Lösel, F., & Loeber, R. (2011). Do the victims of school bullies tend to become depressed later in life? A systematic review and meta-
analysis of longitudinal studies. Journal of Aggression, Conict and Peace Research, 3,6373. http://dx.doi.org/10.1108/17596591111132873.
Van Goethem, A. A., Scholte, R. H., & Wiers, R. W. (2010). Explicit-and implicit bullying attitudes in relation to bullying behavior. Journal of Abnormal Child
Psychology, 38(6), 829842. http://dx.doi.org/10.1007/s10802-010-9405-2.
Waasdorp, T. E., Bradshaw, C. P., & Leaf, P. J. (2012). The impact of schoolwide positive behavioral interventions and supports on bullying and peer rejection: A
randomized controlled eectiveness trial. Archives of Pediatrics & Adolescent Medicine, 166, 149156. http://dx.doi.org/10.1001/archpediatrics.2011.755.
Wang, W. (2013). Bullying among U.S. school children: An examination of race/ethnicity and school-level variables on bullying. All Dissertations (Paper 1204).
Weisberg, H. I. (1979). Statistical adjustments and uncontrolled studies. Psychological Bulletin, 86, 11491164. http://dx.doi.org/10.1037/0033-2909.86.5.1149.
Wilkinson, L., & The Task Force on Statistical Inference (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54,
594604. http://dx.doi.org/10.1037/0003-006X.54.8.594.
Zhang, A., Musu-Gillette, L., & Oudekerk, B. A. (2016). Indicators of school crime and safety: 2015. Washington, DC: National Center for Education Statistics, U.S.
Department of Education, and Bureau of Justice Statistics, Oce of Justice Programs, U.S. Department of Justice.
Zych, I., Tto, M. M., & Farrington, D. P. (2017). Empathy and callousunemotional traits in dierent bullying roles: A systematic review and meta-analysis. Trauma,
Violence, & Abuse (doi: 1524838016683456).
S.P. Limber et al. Journal of School Psychology 69 (2018) 56–72
72
... We focus on the effectiveness of OBPP in the United States (see also Chapter 22). This has fairly consistently demonstrated positive impacts in the United States, including in South Carolina (Limber et al., 2004), Pennsylvania (Limber et al., 2018;Olweus et al., 2019), Washington (Bauer et al., 2007), and Virginia (Farrell et al., 2018). Early results tended to vary and showed less positive effects compared to studies conducted outside of the United States (e.g., Evans et al., 2014;Ttofi & Farrington, 2011); this finding raised some concerns regarding the cultural and contextual fit between OBPP and US schools. ...
... However, recent findings suggest a more promising effect of the OBPP in the United States. Limber et al. (2018) reported on a large-scale study in Pennsylvania across two years (in 210 schools) and three years (in 95 schools). An extended agecohort design was used, in which the scores of a previous cohort of students are used as the untreated comparison group for a subsequent cohort of students who did receive the intervention (Olweus, 2005, and Chapter 22). ...
... Analyses aggregated the data up to the school level, due to the survey being administered anonymously, and therefore the researchers not being able to link an individual child's scores across time. Limber et al. (2018) reported decreases in the aggregated school levels of being bullied and bullying others (school-level effect size [d(S)] varied across grades: d [S] being bullied = 1. 05-2.20; d[S] bullying others = 0.75-1.30), ...
Chapter
Research over the past decade has revealed several major improvements in the effectiveness of bullying prevention programs. Meta-analyses just 10 years ago suggested that the majority of effects of bullying prevention programs were not large enough to be meaningful (Merrell et al., 2008). Moreover, the extant research suggested that bullying prevention programs were less effective in the United States than programs in Europe (Ttofi & Farrington, 2011). However, a more recent updated meta-analysis of bullying prevention programs from a global perspective found that evaluations conducted in North America (the United States and Canada) had the largest effect sizes for the reduction of bullying perpetration compared to other regions (Gaffney et al., 2019). These exciting research developments suggest that when these evidence-based bullying prevention practices are implemented at scale, they have the potential to produce a significant population-level impact (Fagan et al., 2019). This chapter focuses on newer developments in research and evaluation of bullying prevention programs in the United States and Canada, offering suggestions for more effective evaluation and pointing out directions for growth and improvement in bullying prevention strategies.
... Indeed, most interventions are not explicitly designed for at-risk subpopulations (Earnshaw et al., 2018;Xu et al., 2020), and there is a lack of evidence of the effectiveness of traditional antibullying programs on specific subtypes of bullying. Specifically, only two studies analyzed the effects of one antibullying program, the Olweus Bullying Prevention Program (OBPP), separately for native students and youths from ethnic minority groups (Bauer et al., 2007;Limber et al., 2018). The first study proposed a controlled trial with a cohort of 10 middle schools (Grades 6-8). ...
... For the second study, an extended population of 210 schools situated in Pennsylvania was followed over the course of 2 years. The findings showed that although the program had a stronger effect in reducing bullying and victimization within majority groups, the impact was still significant among minority ones (Limber et al., 2018). Although these findings may be due to the considerably smaller numbers of ethnic minority students compared with their native counterparts (Xu et al., 2020), they also underline the need to better study the effects of antibullying programs in subpopulations, and design interventions aimed specifically at preventing ethnic bullying (Limber et al., 2018). ...
... The findings showed that although the program had a stronger effect in reducing bullying and victimization within majority groups, the impact was still significant among minority ones (Limber et al., 2018). Although these findings may be due to the considerably smaller numbers of ethnic minority students compared with their native counterparts (Xu et al., 2020), they also underline the need to better study the effects of antibullying programs in subpopulations, and design interventions aimed specifically at preventing ethnic bullying (Limber et al., 2018). ...
Article
Nowadays, an increasing number of children and adolescents living in Europe have an immigrant background. Because ethnicity is a recognizable characteristic that may become the driver of bullying, these youths are at high risk of victimization. School interventions based on peer-led approaches, assuming all the conditions postulated in contact theory, could be suitable to counteract bias-based bullying and victimization.This study aims to analyze whether the NoTrap! antibullying program, an evidence-based peer-led intervention, may also be effective in counteracting ethnic bullying and victimization when students with an immigrant background are involved as peer educators. There were 1,570 students who participated in the study: 24 control classes (N = 476) and 50 experimental classes (N = 1094). Within the last group we identified two conditions: 30 classes in which all peer educators were Italian (Experimental A, N = 661); 20 classes in which at least one of them had an immigrant background (Experimental B, N = 433). Results of two linear mixed models showed a significant interaction Time × Experimental condition for ethnic victimization, F(1, 1170) = 4.185; p = .015. Specifically, the NoTrap! is effective in reducing ethnic victimization when at least one student with an immigrant background is involved as a peer educator. Only in this condition, indeed, peer educators are directly involved in the phenomenon they counteract, and all four circumstances postulated in the contact theory are satisfied. No effects on ethnic bullying have been found, F(1, 1162) = .215; p = .806. This is in line with the activities proposed in the program, which is more focused on empowering victims than on acting directly on bullies. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
... Teachers can employ such authority-based interventions to send a clear message of disapproval (Saarento et al., 2013), which might reduce bullying by increasing moral engagement in peers (Campaert et al., 2017). Disciplinary sanctions are not only an integral part of many anti-bullying programs, such as the KiVa anti-bullying program , the Olweus bullying prevention program (Limber et al., 2018) and the ViSC social competence program , but their effectiveness is corroborated by a recent meta-analysis on the effectiveness of intervention components in anti-bullying programs (Gaffney et al., 2021). ...
... This result is in line with previous studies showing that disciplining the bully might be an effective measure to reduce bullying (Gaffney et al., 2021). The found associations support the notion that disciplinary sanctions should be considered an essential component of anti-bullying interventions Limber et al., 2018;Strohmeier et al., 2021). It must be pointed out that in the present study disciplinary measures were operationalized of low to moderate severity and more severe measures (e.g., detention, suspension, and expulsion) were not included. ...
Article
Full-text available
School bullying is a serious problem worldwide, but little is known about how teacher interventions influence the adoption of bullying-related student roles. This study surveyed 750 early adolescents (50.5% female; average age: 12.9 years, SD = 0.4) from 39 classrooms in two waves, six months apart. Peer ratings of classmates were used to categorize students to five different bullying-related roles (criterion: >1 SD): bully, victim, bully-victim, defender, and non-participant. Student ratings of teachers were used to obtain class-level measures of teacher interventions: non-intervention, disciplinary sanctions, group discussion, and mediation/victim support. Controlling for student- and class-level background variables, two multilevel multinomial logistic regression analyses were computed to predict students’ bullying-related roles at wave 2. In the static model, predictors were teacher interventions at wave 1, and in the dynamic model, predictors were teacher intervention changes across time. The static model showed that disciplinary sanctions reduced the likelihood of being a bully or victim, and group discussion raised the likelihood of being a defender. Mediation/victim support raised the likelihood of being a bully. The dynamic model complemented these results by indicating that increases in group discussion across time raised the likelihood of being a defender, whereas increases in non-intervention across time raised the likelihood of being a victim and reduced the likelihood of being a defender. These results show that teacher interventions have distinct effects on students’ adoption of bullying-related roles and could help to better target intervention strategies. The findings carry practical implications for the professional training of prospective and current teachers.
... The OBPP is a multi-level, multi-component whole-school approach anti-bullying programme which is built upon a solid evidence foundation and is an internationally recognised programme, demonstrating a positive effect in reducing school bullying (Limber, Olweus, Wang, Masiello & Breivik, 2018;Olweus et al., 2021). A number of evaluations of the OBPP have documented a substantial reduction in bullying problems after eight to nine months of work with the programme, as well as long-term school level effects up to eight years after original implementation (Limber et al., 2018;Olweus et al., 2021). ...
... The OBPP is a multi-level, multi-component whole-school approach anti-bullying programme which is built upon a solid evidence foundation and is an internationally recognised programme, demonstrating a positive effect in reducing school bullying (Limber, Olweus, Wang, Masiello & Breivik, 2018;Olweus et al., 2021). A number of evaluations of the OBPP have documented a substantial reduction in bullying problems after eight to nine months of work with the programme, as well as long-term school level effects up to eight years after original implementation (Limber et al., 2018;Olweus et al., 2021). Moreover, the OBPP is not a "programme" in the narrow sense of this term, but rather a coordinated collection of research-based components that form a unified wholeschool approach to bullying combined with selective interventions (Olweus et al., 2021, p. 412). ...
Chapter
Full-text available
The academic literature suggests that teachers' perceptions of addressing bullying are linked to their actual practices of preventing and intervening in bullying incidents. The current study extends this statement by investigating the relationship between teachers' perceptions of, and their practices within, the various components of an anti-bullying programme at school, classroom and individual levels. The study also aims to investigate if there are differences between teachers' perceptions and practices within an anti-bullying programme and individual characteristics of the teachers. Teachers, working within the Olweus Bullying Prevention Programme in Norway (N = 82), completed a standardised online self-administered questionnaire. The results showed that teachers had a slightly more positive perception and put more effort into preventing bullying at the individual level in comparison to classroom and school levels of the programme. Primary education teachers generally put more effort into working with the anti-bullying programme than lower secondary education teachers. Female teachers put more effort into organisation of Olweus class meetings and 20 INQUIRY AS A BRIDGE IN TEACHING AND TEACHER EDUCATION in following up bullying cases, and they had more positive perceptions of the effectiveness of the anti-bullying programme in comparison to male teachers. Finally, the study indicated a relationship between teachers' perceptions and practices within an anti-bullying programme.
... KiVa Hollanda (Huitsing et al., 2020), Şili (Gaete et al., 2017), Galler, Birleşik Krallık (Axford et al., 2020), Yeni Zelanda (Green et al., 2020) ve Fransa (Garandeau & Salmivalli, 2018). OBPP, Latin Amerika'da (Limber et al., 2018) (Olweus & Limber, 2009) ve Almanya'da (Ossa et al., 2021) kullanılmaktadır. (Smith, 2016). ...
Article
Schools are social environments where students learn to socialize and participate in society. While the school environment aims to enhance the student socially and psychologically, there can be some problems in achieving these goals. One of the biggest issues is peer bullying. Bullying is the practice of physical and psychological pressure from larger individuals or groups that are repeated to less powerful individuals and groups (Olweus, 1993). There is a systematic misuse of perseverance and power in bullying, and there is also a power imbalance (Rigby, 1999). The frequency of bullying varies depending on the country, culture, student age and the measurement tools used. In addition to the role of the student's personality, family and cultural characteristics in bullying behaviors; The structure and size of the school, the attitudes of teachers and administrators, and the school environment have an impact. Worldwide, bullying is on the school education agenda. According to UNICEF 2020 data, half of the world's youth are victims of peer bullying in and around school. Although anti-bullying programmes have been discussed for some time, this issue has recently been raised in Turkey. The statistics in our research will show that bullying affects many students in our country. Although included in the counselling programmes in Turkey, a specific preventive programme is not used. The aim of this study is to compare school-based programs, which are considered to be successful in the world, and to examine their characteristics. Keywords: Bullying, KiVa , OBBP, ViSC, NoTrap
... Based on previous studies, Limber et al. [28] found that evaluation has great impact on both teaching, writing, and students' learning of writing by observing and investigating undergraduates' opinions on writing tests and their reflections after taking writing classes. Klimova [29] found the shortcomings of line evaluation and carried out reforms to 2 Computational Intelligence and Neuroscience further refine the evaluation criteria, including the validity of logical points of discourse, persuasive arguments, and other factors into the evaluation. ...
Article
Full-text available
College English classroom teaching evaluation is one of the key issues discussed by all schools at present. First of all, teachers and students are highly concerned about English learning in the compulsory course of English examination, but there are also many problems. The distance network teaching with computer technology as the main body develops rapidly, a large number of video and audio teaching resources network through the network transmission to present in front of learners, through the network transmission of video and audio aided, expand the audience of network teaching, is conducive to the realization of digitalization, information, lifelong, new education goals. Firstly, this article summarizes the problems of poor effect, uneven performance, and mismatch between evaluation and teaching in college English at present. Then, based on the previous works, the author designed a machine learning-based Internet of things technology and verifies its feasibility. The data of teaching experiments show that the writing performance of the students in the lower section of the experimental class is better than that of the students in the middle and high sections. Among them, significant progresses have been made in grammar, length, expression, and structure, which has optimized students’ preclass preview efficiency, autonomous learning motivation, quality of homework completion, and afterclass reflection behavior. Finally, the author summarized the shortcomings of this research and puts forward the prospect of relevant research, in order to provide reference value for the national college English blended teaching and promotes the efficient implementation and vigorous development of college English teaching. In the future, the evaluation index system under the pressure of performance can be further studied.
Article
Full-text available
Bullying is a severe problem that affects students’ mental health. The number of bullying incidents in Indonesia is still high, most likely because the country has not had bullying prevention and intervention programs implemented holistically in all aspects at school. Indonesia’s bullying prevention and intervention programs have been primarily implemented sectorally, only for certain students when incidents have been observed. Two bullying prevention intervention programs that have been tested multinational, the Olweus Bullying Prevention Program and KiVa, have been proven effective. The aims of this study are twofold: to explain the effectiveness and evaluation of these two stated programs and to give insights for Indonesian researchers interested in developing comprehensive bullying intervention and prevention programs. Discussions are focused on developing the prevention and intervention programs, the implementation of the programs, and the evaluation to test program effectiveness. Results show the importance of involving all school components, integration with the school curriculum, national implementation, and using the programs over an extended period. Evaluation of the programs’ effectiveness is also essential, with a randomized controlled trial recommended for doing so.
Article
Full-text available
A kutatás egyetemi hallgatók (N = 251) körében vizsgálta a cyberbullying és a kiégés jelenségét, továbbá ezek kapcsolatát az önértékeléssel és a tanulmányi motivációval. A résztvevők – 18 és 47 év közötti, nappali, levelező és esti tagozatos hallgatók – egy online kérdőívcsomagot töltöttek ki. Az eredmények alapján egyetemi közegben alkalmanként bevonódott internetes zaklatásba a kitöltők 8,9%-a. A korábbi kutatásokkal összhangban az áldozati és a zaklatói szerep között összefüggés van, nemi különbség csak a zaklatói alskálán mutatkozott. A cyberbullying és a kiégés között pozitív együttjárást találtunk. A hallgatói kiégés súlyosabb tünetei a kitöltők 14,4%-át érintik, az eredmények alapján a teljes skálán nem volt szignifikáns nemi különbség, míg az érzelmi kimerülés alskálán a nők, a hatékonyságcsökkenés alskálán a férfiak értek el magasabb pontszámot. Az eredmények alátámasztják az alacsony önértékelés kapcsolatát a cyberbullyinggal és a kiégés-szindrómával egyaránt. A kiégést együttesen bejósló tényezők az alacsony önértékelés, az intrinzik motiváció hiánya és az amotiváció, az utóbbi önmagában is előrejelző lehet.
Chapter
In less than two decades, social media has become a part of life for many people, increasing the positive and negative effects of social relationships. In a sense, the traditional offline behaviors moved into a ubiquitous environment, enhancing the social phenomenon of cyberbullying. Thus, it attracted much attention from different fields within academia. To understand how research has been conducted in the last 20 years and the topics addressed, this study applied a bibliometric analysis to academic literature from 2000 to 2020 related to cyberbullying in social media, using techniques such as citation analysis, co-citation analysis, and content analysis. The growing interest of the field is confirmed, and some research gaps are unveiled. Since this is one of the first studies to explore cyberbullying that occurs in social media networks, it is hoped that this chapter will stimulate further research on this topic reinforcing the gaps found.
Research
Full-text available
This report covers topics such as victimization, teacher injury, bullying and cyber-bullying, school conditions, fights, weapons, availability and student use of drugs and alcohol, student perceptions of personal safety at school, and criminal incidents at postsecondary institutions. Indicators of crime and safety are compared across different population subgroups and over time. Data on crimes that occur away from school are offered as a point of comparison where available.
Article
Full-text available
Bullying is becoming an ever more pressing issue for schools, daycare centers, politicians and the public. Everyone agrees that bullying is a serious problem and initiatives are urgently called for to stamp it out. This Campbell Systematic Review studied the effects of anti‐bullying programs in schools. The conclusion is that programs generally work and bullying is reduced on average by around 20%. A total of 89 reports were of sufficient quality to be included in the systematic review. The 89 reports describe 53 different studies. However, nine studies did not provide enough data to allow the calculation of an effect size and were, therefore, not included in the final meta‐analysis. The overall analysis is therefore based on a total of 44 studies. The 44 different studies were carried out between 1983 and mid‐2009 and came from 16 different countries. The included studies were either randomized controlled trials, quasi‐randomized trials, age‐cohort studies or other controlled studies. Furthermore, the systematic review clearly states that future evaluations should measure the children's situation before and after an anti‐bullying program. This should apply to the experimental group as well as the control group to get the most accurate results possible. Executive Summary/Abstract BACKGROUND School bullying has serious short‐term and long‐term effects on children's physical and mental health. Various anti‐bullying programs have been implemented world wide and, more rarely, evaluated. Previous narrative reviews, summarizing the work done on bullying prevention, as well as previous meta‐analyses of anti‐bullying programs, are limited. The definition of school bullying includes several key elements: physical, verbal, or psychological attack or intimidation that is intended to cause fear, distress, or harm to the victim; an imbalance of power (psychological or physical), with a more powerful child (or children) oppressing less powerful ones; and repeated incidents between the same children over a prolonged period. School bullying can occur in school or on the way to or from school. It is not bullying when two persons of the same strength (physical, psychological, or verbal) victimize each other. OBJECTIVES This report presents a systematic review and meta‐analysis of the effectiveness of programs designed to reduce school bullying perpetration and victimization (i.e. being bullied). The authors indicate the pitfalls of previous reviews and explain in detail how the present systematic review and meta‐analysis addresses the gaps in the existing literature on bullying prevention. SEARCH STRATEGY In the present report, we go beyond previous reviews by: doing much more extensive searches for evaluations such as hand‐searching all volumes of 35 journals from 1983 up to the end of May 2009; searching for international evaluations in 18 electronic databases and in languages other than English; and focusing only on programs that are specifically designed to reduce bullying and not aggressive behavior (i.e. the outcome variables specifically measure bullying). Leading researchers in the area of school bullying were also contacted via e‐mail. SELECTION CRITERIA Studies were included in this review if they evaluated the effects of an anti‐bullying program by comparing an experimental group who received the intervention with a control group who did not. The word ‘experimental’ here refers to students who received the program and does not necessarily imply randomization. Four types of research design were included: a) randomized experiments, b) experimental‐control comparisons with before and after measures of bullying, c) other experimental‐control comparisons and d) quasi‐experimental age‐cohort designs, where students of age X after the intervention were compared with students of the same age X in the same school before the intervention. Both published and unpublished (e.g. PhD theses) reports were included. Reports concerning an evaluation of a program had to clearly indicate that bullying or victimization were included as outcome measures. Bullying and victimization could be measured using self‐report questionnaires, peer ratings, teacher ratings, or observational data. RESULTS We found a total of 622 reports that were concerned with bullying prevention. The number of reports on anti‐bullying programs and on the necessity of tackling bullying has increased considerably over time. Only 89 of these reports (describing 53 different program evaluations) could be included in our review. Of the 53 different program evaluations, only 44 provided data that permitted the calculation of an effect size for bullying or victimization. Our meta‐analysis of these 44 evaluations showed that, overall, school‐based anti‐bullying programs are effective in reducing bullying and victimization (being bullied). On average, bullying decreased by 20% – 23% and victimization decreased by 17% – 20%. The effects were generally highest in the age‐cohort designs and lowest in the randomized experiments. It was not clear, however, that the randomized experiments were methodologically superior in all cases, because sometimes a very small number of schools (between three and seven) were randomly assigned to conditions, and because of other methodological problems such as differential attrition. Various program elements and intervention components were associated with a decrease in both bullying and victimization. Work with peers was associated with an increase in victimization. We received feedback from researchers about our coding of 40 out of 44 programs. Analyses of publication bias show that the observed effect sizes (for both bullying and victimization) were based on an unbiased set of studies. AUTHORS’ CONCLUSIONS Results obtained so far in evaluations of anti‐bullying programs are encouraging. The time is ripe to mount a new long‐term research strategy on the effectiveness of these programs, based on our findings. The main policy implication of our review is that new anti‐bullying programs should be designed and tested based on the key program elements and evaluation components that we have found to be most effective. We recommend that a system of accrediting anti‐bullying programs should be developed, supervised by an international body such as the International Observatory on Violence in Schools.
Article
Full-text available
Impetus for the intervention study, early stages of planning, and funding In 1994 the Institute for Families in Society at the University of South Carolina received a grant from the Office of Juvenile Justice and Delinquency Prevention (within the US Department of Justice) to undertake a 3-year project to conduct research related to violence among rural youth. A critical component of this grant was the implementation and evaluation of the Olweus Bullying Prevention Programme in rural schools in South Carolina. Faculty at the Institute for Families in Society were familiar with the success of the programme in Norwegian schools, from reading published summaries of the programme (Olweus, 1991; 1993) and from discussions with Professor Olweus. Recognising that no other violence-prevention programme to date had produced such impressive results, faculty were anxious to test the programme in an American setting. Early planning and preparation for the implementation of the programme involved the hiring of staff, extensive consultation with Professor Olweus, and the selection and preparation of participating schools. With the receipt of the Justice Department grant, the Institute hired a project director and part-time faculty, staff, and graduate students to support the implementation of the project. The principal university project team consisted of the principal investigator (Melton), a three-quarter time project director (Limber), three part-time PhD-level faculty at the Institute and at the Medical University of South Carolina who were responsible for providing ongoing technical assistance to schools, a part-time consultant for the project’s evaluation, and several graduate research assistants to collect, input, and collate data from participating schools.
Article
Full-text available
In this paper, we review and critique current bully prevention efforts. This work presents an integrative, narrative review of bully prevention and school climate improvement research. We suggest that targeted bully prevention programs and curricular-based efforts are marginally helpful. We suggest that a continuous process of school climate improvement needs to be an integral and organizing anchor for effective bully prevention and school improvement efforts. Based on our collective experience with complex group phenomena, as well as an understanding of recent school improvement and violence prevention research, we outline a series of comprehensive, research-based guidelines that subsume targeted efforts within more comprehensive approaches that promote positive, sustained school climates. Such approaches not only prevent bully-victim-bystander behavior but also promote safe, supportive, responsible, engaging, flourishing and democratically informed school communities.
Article
Bullying is an extremely damaging type of violence that is present in schools all over the world, but there are still many gaps in knowledge regarding different variables that might influence the phenomenon. Two promising research lines focus on empathy and callous–unemotional traits but findings from individual studies seem to be contradictory. This article reports the results of a systematic review and a meta-analysis on empathy and callous–unemotional traits in relation to school bullying based on 53 empirical reports that met the inclusion criteria. Bullying perpetration is negatively associated with cognitive (odds ratio [OR] = 0.60) and affective (OR = 0.51) empathy. Perpetration is also positively associated with callous–unemotional traits (OR = 2.55). Bully-victims scored low in empathy (OR = 0.57). There is a nonsignificant association between victimization and empathy (OR = 0.96), while the relationship between callous–unemotional traits and victimization is significant but small (OR = 1.66). Defenders scored high on cognitive (OR = 2.09) and affective (OR = 2.62) empathy. These findings should be taken into account in explaining and preventing bullying.