Content uploaded by Mauricio Herrera López
Author content
All content in this area was uploaded by Mauricio Herrera López on Jan 14, 2020
Content may be subject to copyright.
Content uploaded by Mauricio Herrera López
Author content
All content in this area was uploaded by Mauricio Herrera López on Mar 25, 2017
Content may be subject to copyright.
Validation of the European Cyberbullying Intervention
Project Questionnaire for Colombian Adolescents
Mauricio Herrera-Lo´ pez, PhD,
1,
*Jose´ A. Casas, PhD,
2
Eva M. Romera, PhD,
2
Rosario Ortega-Ruiz, PhD,
2,3
and Rosario Del Rey, PhD
4
Cyberbullying is the act of using unjustified aggression to harm or harass via digital devices. Currently regarded
as a widespread problem, the phenomenon has attracted growing research interest in different measures of
cyberbullying and the similarities and differences across countries and cultures. This article presents the
Colombian validation of the European Cyberbullying Intervention Project Questionnaire (ECIPQ) involving
3,830 high school students (M=13.9 years old, standard deviation =1.61; 48.9 percent male), of which 1,931
were Colombian and 1,899 Spanish. Confirmatory factor analysis (CFA), content validation, and multigroup
analysis were performed with each of the sample subgroups. The optimal fits and psychometric properties
obtained confirm the robustness and suitability of the assessment instrument to jointly measure cyber-
aggression and cyber-victimization. The results corroborated the theoretical construct and the two-dimensional
and universal nature of cyberbullying. The multigroup analysis showed that cyberbullying dynamics are similar
in both countries. The comparative analyses of prevalence revealed that Colombian students are less involved in
cyberbullying. The results indicate the suitability of the instrument and the advantages of using such a tool to
evaluate and guide psychoeducational interventions aimed at preventing cyberbullying in countries where few
studies have been performed.
Keywords: Colombia, cyberbullying, ECIPQ, Spain, validation
Introduction
The prevalent use of information and communications
technologies (ICTs) has significantly transformed in-
terpersonal relationships among adolescents.
1
Although ICTs
have certain benefits, they have also given rise to a complex
scenario of interactions that require new abilities and social
skills to navigate cyberspace successfully.
2
Research has also
shown that the use of ICTs has led to an increase in social
problems, including cyberbullying;
3
a phenomenon currently
regarded as a major public health issue in schools
4,5
given its
negative impact on the social and emotional development of
children and adolescents.
1,6,7
It is estimated that around 20
percent of young people aged from 10 to 18 have been cy-
berbullies or cybervictims;
8,9
with puberty and adolescence
increasing the risk of becoming involved in cyberbullying.
10
Cyberbullying is an aggressive, intentional act or act of
intimidation carried out using electronic media, which cre-
ates an imbalance of power between the bully and the vic-
tim.
11,12
Some authors regard this phenomenon as an indirect
form of harassment, as it is conceived within the definitional
framework of traditional bullying. Certain characteristics,
however, differentiate cyberbullying from traditional bully-
ing, such as the potential and, in some cases, frequent ano-
nymity of the perpetrator,
13,14
and the duration of an aggressive
act in cyberspace, where an image or other humiliating au-
diovisual material is freely accessible (is public) at any
time.
15
Insulting or threatening behavior through text mes-
sages or the Internet; spreading rumors about someone on
social networks; extracting, disclosing, or publishing personal
information; displaying or sending compromising photos of
someone; excluding or being excluded from a group or chat;
and online identity theft are all actions regarded as cyber-
bullying.
16
Moreover, it is important to highlight that both
bullying and cyberbullying constitute unjustified behavior
that involves a certain degree of immorality.
17
Although there have been notable advances in research on
cyberbullying over the last decade,important questions remain
with regard to the adequate assessment of the phenomenon.
This may be due to cyberbullying’s multiple manifestations,
1
Department of Psychology, University of Narin
˜o (UDENAR), San Juan de Pasto, Colombia.
2
Department of Psychology, University of Co
´rdoba (UCO), Co
´rdoba, Spain.
3
Department of Psychology, University of Sevilla, Sevilla, Spain.
4
Department of Psychology, Social Work and Counselling, University of Greenwich, London, United Kingdom.
CYBERPSYCHOLOGY,BEHAVIOR,AND SOCIAL NETWORKING
Volume 20, Number 2, 2017
ªMary Ann Liebert, Inc.
DOI: 10.1089/cyber.2016.0414
117
which render it difficult to develop and validate scales with
optimal psychometric properties.
18
A systematic review of
636 studies showed that most of the existing measurement
instruments do not take into account the properties and theo-
retical and structural factors involved, and of the 44 recognized
instruments, only 24 reported convergent validity.
19
Other
studies have shown that the scales used to measure cyberbul-
lying do not have the same factorial structure, unlike tradi-
tional instruments for assessing bullying.
20
In developing an
instrument to measure cyberbullying, Law et al.
21
found that
the items used to assess traditional bullying were clearly or-
ganized into two factors (victimization and aggression), while
items related to cyberbullying were organized into just one.
The same result was obtained by Menesini et al.,
22
who re-
ported a tendency for single factor grouping. In general, there
are few instruments that comprehensively identify and mea-
sure all the factors related to cyberbullying involvement.
18,23
Indeed, it is still very common to find studies that measure only
cyber-aggression
21,24
or cyber-victimization,
25
thus increas-
ing the propensity of biased measures.
26
The European Cyberbullying Intervention Project Ques-
tionnaire (ECIPQ)
18
was designed based on the studies of
Dooley et al.
27,28
This rigorous measurement instrument is
comparable to other international instruments, and it has
been validated in six European countries with optimal psy-
chometric results.
14–30
The ECIPQ includes new definitions
of cyberbullying, reflects its various manifestations and
recognizes the dynamic structure of the phenomenon by
measuring its two principal dimensions: aggression and
victimization. It also makes no distinction between digital
devices (mobile phones and PCs) and includes the criteria of
repetition and the imbalance of power as it assesses the do-
minion of technology in the aggression and the lack of se-
curity measures in victimization.
18
A recent worldwide review of cyberbullying studies has
shown that the articles published on the Web of Science are
mainly studies from North America (57 percent) and Europe
(28 percent), followed by Western Asia (8 percent) and
Australia (4 percent), which, and denoting an uneven geo-
graphical distribution, highlights the scarce scientific par-
ticipation of developing countries, such as those in Latin
America.
29
The small number of studies conducted in these
countries tend to report data on prevalence, primarily using
instruments lacking rigorous psychometric properties. Other
studies report on the type of harassment and the description
and characterization of cyberbullying,
30,31
while very few
examine the development or validation of instruments.
32–34
This situation, coupled with the scarcity of cross-cultural
research, represents important limitations for advancing in
the study of cyberbullying in regions such as Latin America,
where violence is considered to be even more exacerbated.
35
The validation and use of a common instrument with known
psychometric properties to compare the phenomenon of cy-
berbullying would allow us to get further insight into the
universality of the phenomenon and examine differences and
similarities between countries; additionally comparisons
between two different geographical regions, which share a
similar culture, history, and same language, are of particular
interest as they open up new lines of research into cyber-
bullying beyond cross-cultural, comparative studies among
European countries
18
or between the United States and
countries of Asia.
36
Owing to the limited availability of adequately validated
instruments for Latin American countries and the need for
internationally recognized instruments incorporating both
cyber-victimization and cyber-aggression, the main objec-
tive of this study was to validate the ECIPQ
18
scale in Co-
lombia. The second objective was to compare the prevalence
of involvement in cyberbullying in both countries (Colombia
and Spain) focusing on the following roles: bullies, victims,
bully-victims, and noninvolved students. Our hypothesis was
that the instrument would show a two-factor structure with
optimal psychometric properties and measurement homo-
geneity in the Spanish version, while Colombian students
would be less involved in cyberbullying based on the as-
sumption that technology is used to a lesser degree in the
country.
37
Methodology
Participants
The overall sample comprised 3,830 adolescent students
(48.9 percent male) aged 10 to 19 (M=13.95, standard de-
viation [SD]=1.61). The Colombian subgroup consisted of
1,931 students (46.9 percent male) of public and private
schools in the city of San Juan de Pasto, southern Colombia,
aged 10 -19 (M=14.9; SD =1.89); while the Spanish sub-
group consisted of 1,899 students (51 percent male) of public
and private schools in Andalucia, southern Spain, aged 11 -
18 (M=14.3; SD =1.81).
Instrument
We used the ECIPQ,
18
which comprises 22 items (11 for
cyber-victimization and 11 for cyber-aggression). The
ECIPQ uses a Likert-type scale with five response options
ranging from 0 =never, 1 =once or twice, 2 =once or twice a
month, 3 =about once a week, and 4 =more than one once a
week. An example of an item for cyber-victimization is
‘‘Someone said nasty things to me or called me names using
texts or online messages,’’ while ‘‘I spread rumours about
someone on the Internet’’ is an example of a cyber-
aggression item. The internal consistency of the original test
is optimal: a
cyber-victimization
=0.97; a
cyber-aggression
=0.93;
a
total
=0.96.
Procedure
We used a cross-sectional, ex post facto, retrospective, one
group, one measurement research design.
38
To ensure ethical
standards, we first obtained authorization from the school
officials and subsequently sent informed consent forms to the
students’ parents and/or legal guardians. After receiving
signed consent from the parents, we then visited the students
who had been authorized to take part in the study and asked
them to sign a consent form. Before administering the
questionnaire to the students, we informed them that par-
ticipation was anonymous and voluntary and explained the
objective of the study. The average time taken to complete
the questionnaire was 15 minutes. Convenience sampling
was performed owing to accessibility.
The scale content was validated by a panel of six Co-
lombian experts, who were given the Spanish version
translated from English to Castilian by Ortega-Ruiz et al.
39
with the parallel back-translation procedure.
40
The panel
118 HERRERA-LO
´PEZ ET AL.
assessed the compliance of criteria such as adequacy of vo-
cabulary, conceptual clarity, and the consistency and rele-
vance of each item. To this end, a 4-point scale was used
where 1 =noncompliance, 2 =low level of compliance,
3=moderate compliance, and 4 =high level of compliance.
Finally, a pilot test was conducted with 60 students to assess
their understanding of the items. Any words or terms in the
original Spanish text that were not clear to the students were
modified. For example, the item ‘‘Alguien me ha dicho pa-
labras malsonantes o me ha insultado usando el e-mail o
FIG. 1. CFA of the adapted
ECIPQ for Colombia
(*p<0.05). CFA, confirma-
tory factor analysis; ECIPQ,
European Cyberbullying
Intervention Project
Questionnaire.
VALIDATION OF THE ECIPQ FOR COLOMBIA 119
Table 1. Polychoric Correlation Matrix: European Cyberbullying Intervention Project Questionnaire
Items 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
11
2 0.77 1
3 0.57 0.57 1
4 0.36 0.40 0.45 1
5 0.37 0.39 0.44 0.80 1
6 0.33 0.40 0.40 0.44 0.55 1
7 0.40 0.43 0.51 0.46 0.49 0.60 1
8 0.32 0.39 0.43 0.41 0.38 0.42 0.67 1
9 0.39 0.38 0.42 0.48 0.41 0.37 0.66 0.64 1
10 0.36 0.39 0.37 0.30 0.30 0.29 0.40 0.44 0.48 1
11 0.59 0.62 0.48 0.36 0.37 0.41 0.58 0.42 0.48 0.40 1
12 0.58 0.50 0.43 0.32 0.32 0.36 0.46 0.40 0.45 0.39 0.52 1
13 0.52 0.47 0.42 0.29 0.36 0.27 0.44 0.44 0.41 0.34 0.51 0.80 1
14 0.39 0.42 0.62 0.35 0.35 0.48 0.54 0.51 0.57 0.42 0.45 0.62 0.65 1
15 0.38 0.35 0.45 0.38 0.35 0.48 0.48 0.44 0.32 0.20 0.29 0.45 0.52 0.64 1
16 0.19 0.34 0.45 0.39 0.41 0.53 0.52 0.55 0.41 0.37 0.33 0.51 0.50 0.67 0.80 1
17 0.27 0.35 0.38 0.36 0.32 0.47 0.42 0.48 0.40 0.38 0.35 0.35 0.46 0.59 0.58 0.71 1
18 0.37 0.43 0.49 0.47 0.43 0.55 0.69 0.73 0.70 0.55 0.56 0.55 0.58 0.71 0.56 0.71 0.69 1
19 0.34 0.42 0.49 0.42 0.40 0.51 0.58 0.72 0.60 0.49 0.48 0.48 0.53 0.61 0.53 0.65 0.62 0.79 1
20 0.29 0.36 0.41 0.31 0.36 0.40 0.47 0.51 0.62 0.44 0.44 0.51 0.48 0.59 0.51 0.66 0.53 0.78 0.74 1
21 0.25 0.34 0.34 0.23 0.28 0.35 0.42 0.37 0.40 0.37 0.40 0.49 0.47 0.56 0.48 0.54 0.42 0.56 0.51 0.57 1
22 0.33 0.43 0.41 0.28 0.33 0.42 0.44 0.60 0.57 0.43 0.60 0.52 0.60 0.52 0.45 0.56 0.46 0.70 0.66 0.65 0.56 1
All correlations with p<0.01.
120
SMS’’ (Someone used bad words or insulted me using e-mail
or SMS) was substituted for ‘‘Alguien me ha dicho groserı
´as
o insultado por internet (e-mail, redes sociales, llamadas o
SMS)’’ (Someone made rude comments or insulted me on
Internet [e-mail, social networks, calls or SMS]). The
changes were then incorporated into the Colombian version
(Appendix Table A1).
To establish the different roles of involvement, we fol-
lowed the criteria established by the authors of the ECIPQ
scale.
18
For example, to determine the role of cybervictim we
took into account subjects with scores q2 (once a month) in
all the cyber-victimization items, and a score p1 (once or
twice) in all of the cyber-aggression items. Involvement in
the role of cyberbully was measured taking into account
subjects with scores q2 (once a month) in any of the cyber-
aggression items, and scores p1 (once or twice) in all the
cyber-victimization items. The degree of involvement in the
role of bully-victim was obtained with scores q2 (once a
month) in at least one of the cyber-aggression items and in at
least one of the cyber-victimization items. Finally, nonin-
volvement was measured using scores p1 (once or twice) in
all cyber-aggression and cyber-victimization items.
Data analysis
A confirmatory factor analysis (CFA) was performed for
the structural validation of the scale. Maximum likelihood
(ML) estimation with robust correction
41
and polychoric
correlation were used, given the categorical nature of the
variables.
42
To assess the suitability of the instrument, we
used the Satorra-Bentler scaled chi-square test
43
(v
2
S-B
), chi-
square divided by its degrees of freedom (v
2
S-B
/df)(p5
acceptable p3 optimal), the comparative fit index (CFI), and
the non-normed fit index (NNFI) (whose values must be
q0.95).
44,45
We also took into account the root mean square
error approximation (RMSEA p0.05), the standardized root
mean square residual (SRMR p0.08 acceptable, p0.05
optimal),
45
and the Akaike information criterion (AIC),
which is used to compare models (the lowest value indicates
the best model). The analysis was performed using the EQS
6.2 program.
46
A McDonald’s Omega (O) test was performed to analyze
the internal consistency of the instrument given that the
variables were categorical and reflected the absence of
multivariante normality.
47
The analysis was performed using
the FACTOR 9.2 program.
48
To assess the degree of robustness of the factorial structure
and hence the degree of generalization of the model in the
two countries, a multigroup analysis was performed. This
analysis requires a series of sequential comparisons using
progressively restricted models starting with a Model 1 to
test the configurational invariance, conferring the same fac-
torial structure to the two subgroups. To analyze the mea-
surement invariance, we proposed three models: Model 2,
where the covariances in both sample groups were restricted;
Model 3, where the factor loadings were made equal; and
Model 4 in which the residuals were restricted.
49
After de-
veloping the models, the delta values (D) of the NNFI, CFI,
RMSEA, and SRMR measures of fit were obtained using a
variance of p0.01 as the cutoff point to accept the invariance
hypothesis.
50
The chi-square difference test (Dv
2
S-B
) was
also performed, where nonsignificant differences indicate
model invariance.
51
This multigroup analysis was performed
using the EQS 6.2 program.
46
To compare the differences between countries regarding
the roles of involvement included in the questionnaire, we
performed a chi-square test (v
2
), taking into account the
values of the adjusted standardized residuals greater than
–1.96 (95 percent confidence interval [CI]) and –2.58 (99
percent CI).
The level of significance was 0.05.
Results
The validation of the ECIPQ
18
content for Colombia,
based on the assessment of the expert panel, showed an ad-
equate degree of agreement (r
k
=0.81).
The CFA performed with the Colombian subsample in-
dicated that the assumptions of multivariate normality
were not met, as a Mardia coefficient value =875.13 was
obtained. The original two-factor structure was confirmed by
the adequate fit indices (v
2
S-B
=644.97; v
2
S-B
/(208) =3.10;
p<0.001; NNFI =0.97; CFI =0.97; RMSEA =0.047 (90
percent CI [0.043, 0.052]); SRMR =0.080; AIC =228.96)
(Fig. 1 and Table 1). The total internal consistency and the
consistency of each factor were optimal (Ocyber-
aggression =0.94; Ocyber-victimization =0.91; Oto-
tal =0.95).
The CFA performed with the Spanish subsample con-
firmed the original two-factor structure (v
2
S-B
=563.07;
v
2
S-B
/(208) =2.71; p<0.001; NNFI =0.96; CFI =0.97;
RMSEA =0.031 (90 percent CI [0.028, 0.034]); SRMR =
0.079; AIC =147.07). The total internal consistency and the
internal consistency of each factor were also optimal (O
cyber-aggression =0.96, Ocyber-victimization =0.94, O
total =0.97).
The results of the multigroup analysis were within the
established cutoff values (Table 2). In addition, the chi-
square differences between models 1 and 2, 1 and 3, and
models 1 and 4 were not significant. These results
Table 2. Multigroup Analysis: Configuration and Measurement Invariance
Mod v
2
S-B
df p NNFI CFI RMSEA SRMR Dv
2
S-B
DpDdf DNNFI DCFI DRMSEA DSRMR
Mod 1 2,503.97 416 0.00 0.96 0.96 0.05 0.08
Mod 2 2,492.47 436 0.00 0.96 0.96 0.05 0.08 11.50 0.94 (ns) 20 0.00 0.00 0.00 0.00
Mod 3 2,483.36 417 0.00 0.96 0.96 0.05 0.08 20.61 0.97 (ns) 1 0.00 0.00 0.00 0.00
Mod 4 2,537.58 437 0.00 0.96 0.96 0.05 0.09 33.61 0.98 (ns) 21 0.00 0.00 0.00 0.01
CFI, comparative fit index; Mod 1, no restrictions; Mod 2, loaded factorial restrictions; Mod 3, covariance factorial restrictions; Mod 4,
residual restriction; NNFI, non-normed fit index; ns, not significant; RMSEA, root mean square error approximation; SRMR, standardized
root mean square residual.
VALIDATION OF THE ECIPQ FOR COLOMBIA 121
demonstrate the existence of invariance in the factorial
structure of the scale, indicating an optimal degree of ro-
bustness.
The chi-square analysis of roles of involvement indicated
a statistically significant and directly proportional relation-
ship between the country and cyber-aggression, with the
Spanish students being the most involved in this role v
2
(1,
1,873) =21.006; p=0.000, and between the country and
noninvolvement v
2
(1, 1,901) =7.062; p=0.008, with the
Colombian students being the least involved (Table 3).
Discussion
The aim of this study was to validate the ECIPQ scale for
Colombia; an internationally recognized measurement in-
strument of proven psychometric quality.
18
The analysis
confirmed the original two-factor structure of the ECIPQ:
cyber-aggression and cyber-victimization. Optimal values
and fit indices, in addition to good internal consistency were
also obtained. These results support the theoretical construct
that cyberbullying consists primarily of cyber-aggression
and cyber-victimization and that both dimensions are clearly
defined as in traditional bullying.
11
The results for the configuration and measurement in-
variance obtained in the multigroup analysis between Co-
lombia and Spain show that (a) the factorial structural of the
scale presents good robustness, thus ensuring optimal and
rigorous properties, as well as the added benefit of being able
to jointly measure the two major dimensions of cyberbully-
ing, cyber-aggression and cyber-victimization,
36,39,52
and (b)
despite the differences found in the role of aggressor and no
involvement, the dynamics of cyberbullying could be simi-
lar. This similarity could be attributed, among other things,
to the shrinking technology gap.
53
Although this inference
should be taken with caution, it could be a new line of re-
search focused on comparing the cultural aspects and the use
of new information technologies.
A more detailed analysis of the factorial model validated
for Colombia regarding the high factor loadings and satu-
ration in the items relating to rumors (gossip) and misuse of
personal data (pictures and videos) as bully or victim suggest
both the high tendency to engage intimidating practices us-
ing digital devices and the need for more education targeted
at the prevention and management of the personal data that
adolescents divulge in cyberspace.
54,55
The second hypothesis of this study was confirmed by
comparing the prevalence of cyberbullying among Co-
lombian and Spanish students. The results show that the
former are less involved, particularly in the role of cyber-
bully. These results could be attributed to the lower use of
information technologies by Colombian youth. Also may be
due to specific values regarding attitudes, behaviors, and
habits of Colombian adolescents, related to the collectivist
and restrictive Colombian school culture,
54,56
characterized
by respect for the rules of the institutions, conformity, and
obedience.
57
On the contrary, Spanish school culture pro-
motes individualism and self-assertion,
58
which leads them
to greater use of social networks. It has been shown that a
high use increases the risk of involvement in cyberbully-
ing.
59
In conclusion, owing to its optimal psychometric proper-
ties and ability to measure cyber-victimization and cyber-
aggression in a comprehensive manner, the ECIPQ is a
valuable instrument for gaining greater knowledge of cy-
berbullying. It could also be of use in other comparative
studies on the possible differences in the prevalence of cy-
berbullying and associated variables or that assess the results
of interventions targeted at preventing and reducing cyber-
bullying.
This study has some limitations, such as the cross-
sectional nature of the analysis and the potential social desir-
ability bias common to self-reporting.
60
Moreover, although
the overall sample was large, the Colombian subsample was
not representative, which could limit the generalization of the
model. As a future line of research, the sample could include
other Latin American countries, which would allow for
broader cross-cultural studies.
Acknowledgments
We would like to thank members from the schools of San
Juan de Pasto, Colombia, and Andalusia, Spain, for partici-
pating in the study. This study was performed within the
framework of the following projects: Project PRY040/14
funded by the Fundacio
´nPu
´blica Andaluza Centro de Estu-
dios Andaluces, Project EDU2013-44627-P funded by the
Spanish National R&D Plan, and Project BIL/14/S2/163
funded by the Fundacio
´n Mapfre.
Author Disclosure Statement
No competing financial interests exist.
References
1. Kubiszewski V, Fontaine R, Potard C, et al. Does cyber-
bullying overlap with school bullying when taking modal-
ity of involvement into account? Computers in Human
Behavior 2015; 43:49–57.
2. Hymel S, Swearer SM. Four decades of research on school
bullying. An introduction. American Psychologist 2015;
70:293–299.
Table 3. Percentage of Involvement in Cyberbullying
Country
Involvement
Cyber-victimization Cyber-aggression Cyberbully-victim Not involved
Colombia 10.7% SR =0.4 2.5% SR =-4.6 5.5% SR =-1.3 81.0%** SR =2.7
Spain 9.3% SR =-0.4 5.3%*** SR =4.6 6.4% SR =1.3 78.6% SR =-2.7
**p<0.01; ***p <0.001.
SR, standardized residual.
122 HERRERA-LO
´PEZ ET AL.
3. Modecki KL Minchin J, Harbaugh AG, et al. Bullying
prevalence across contexts: a meta-analysis measuring cy-
ber and traditional bullying. Journal of Adolescent Health
2014; 55:602–611.
4. Aboujaoude E, Savage MW, Starcevic V, et al. Cyberbul-
lying: review of an old problem gone viral. Journal of
Adolescent Health 2015; 57:10–18.
5. Selkie EM, Kota R, Chan YF, et al. Cyberbullying, de-
pression, and problem alcohol use in female college stu-
dents: a multisite study. Cyberpsychology, Behavior, and
Social Networking 2015; 18:79–86.
6. Brewer G, Kerslake J. Cyberbullying, self-esteem, empathy
and loneliness. Computers in Human Behavior 2015; 48:
255–260.
7. Casas JA, Del Rey R, Ortega-Ruiz R. Bullying and cy-
berbullying: convergent and divergent predictor variables.
Computers in Human Behavior 2013; 29:580–587.
8. Hinduja S, Patchin JW. Cyberbullying: neither an epidemic
nor a rarity. European Journal of Developmental Psychol-
ogy 2012; 9:539–543.
9. Shin N, Ahn H. Factors affecting adolescents’ involvement
in cyberbullying: what divides the 20% from the 80%?
Cyberpsychology, Behavior, and Social Networking 2015;
18:393–399.
10. Kowalski RM, Giumetti GW, Schroeder AN, et al. Bullying
in the digital age: a critical review and meta-analysis of
cyberbullying research among youth. Psychological Bul-
letin 2014; 4:1073–1137.
11. Olweus D. Cyberbullying: an overrated phenomenon?
European Journal of Developmental Psychology 2012; 9:
520–538.
12. Smith PK. The nature of cyberbullying and what we can do
about it. Journal of Research in Special Educational Needs
2015; 15:176–184.
13. Baldry AC, Farrington D, Sorrentino A. ‘‘Am I at risk of
cyberbullying’’? A narrative review and conceptual
framework for research on risk of cyberbullying and cy-
bervictimization: the risk and needs assessment approach.
Aggression and Violent Behavior 2015; 23:36–51.
14. Wright MF. Predictors of anonymous cyber aggression:
the role of adolescents’ beliefs about anonymity, aggres-
sion, and the permanency of digital content. Cyberpsy-
chology, Behavior, and Social Networking 2014; 17:
431–438.
15. Menesini E, Nocentini A, Palladino BE, et al. Cyberbul-
lying definition among adolescents: a comparison across six
European countries. Cyberpsychology, Behavior, and So-
cial Networking 2012; 15:455–463.
16. Perren S, Corcoran L, Cowie H, et al. Tackling cyberbul-
lying: review of empirical evidence regarding successful
responses by students, parents, and schools. International
Journal of Conflict and Violence 2012; 6:283–292.
17. Ortega-Ruiz R, Zych I. Cyber-behavior and educational
psychology: challenges and risks [in Spanish]. Psicologı
´a
Educativa 2016; 22:1–4.
18. Del Rey R, Casas JA, Ortega-Ruiz R, et al. Structural
validation and cross-cultural robustness of the European
Cyberbullying Intervention Project Questionnaire. Com-
puters in Human Behavior 2015; 50:141–147.
19. Berne S, Frise
´n A, Schultze-Krumbholz A, et al. Cyber-
bullying assessment instruments: a systematic review. Ag-
gression and Violent Behavior 2013; 18:320–334.
20. Schultze-Krumbholz A, Go
¨bel K, Scheithauer H, et al. A
comparison of classification approaches for cyberbullying
and traditional bullying using data from six European
countries. Journal of School Violence 2015; 14:47–65.
21. Law DM, Shapka JD, Hymel S, et al. The changing face of
bullying: an empirical comparison between traditional and
internet bullying and victimization. Computers in Human
Behavior 2012; 28:226–232.
22. Menesini E, Nocentini A, Calussi P. The measurement of
cyberbullying: dimensional structure and relative item se-
verity and discrimination. Cyberpsychology, Behavior, and
Social Networking 2011; 5:267–274.
23. Palladino BE, Nocentini A, Menesini E. Psychometric prop-
erties of the Florence CyberBullying-CyberVictimization
Scales. Cyberpsychology, Behavior, and Social Networking
2015; 18:112–119.
24. Calvete E, Orue I, Este
´vez A, et al. Cyberbullying in ad-
olescents: modalities and aggressors’ profile. Computers in
Human Behavior 2010; 26:1128–1135.
25. Tynes BM, Rose CA, Williams DR. The development and
validation of the Online Victimization Scale for adolescents.
Journal of Psychosocial Research on Cyberspace 2010; 4:1–15.
26. Dempsey AG, Sulkowski ML, Nichols R, et al. Differences
between peer victimization in cyber and physical settings
and associated psychosocial adjustment in early adoles-
cence. Psychology in the Schools 2009; 46:962–972.
27. Dooley JJ, Py_
zalski J, Cross D. Cyberbullying versus face-
to-face bullying. Zeitschrift fu
¨r Psychologie/Journal of
Psychology 2009; 217:182–188.
28. Py_
zalski J. From cyberbullying to electronic aggression:
typology of the phenomenon. Emotional and Behavioural
Difficulties 2012; 17:305–317.
29. Zych I, Ortega-Ruiz R, Del Rey R. Scientific research on
bullying and cyberbullying: where have we been and where
are we going. Aggression and Violent Behavior 2015; 24:
188–198.
30. Del Rı
´o PJ, Bringue SX, Sa
´bada CC, et al. Cyberbullying: a
comparative analysis in students from Argentina, Brazil, Chile,
Colombia, Mexico, Peru and Venezuela [in Spanish]. In: V
Congre
´s Internacional Comunicacio
´I Realitat. Barcelona;
2010:307–316.
31. Mura G, Diamantini D. Cyberbullying among Colombian
students: an exploratory investigation. European Journal of
Investigation in Health 2013; 3:249–256.
32. Buelga S, Cava MJ, Musitu G. Validation of the adolescent
victimization through mobile phone and internet scale [in
Spanish]. Revista Panamericana De Salud Pu
´blica/Pan
American Journal of Public Health 2012; 32:36–42.
33. Baquero CA, Avendan
˜o PBL. Design and psychometric
analysis of an instrument to detect the presence of cyber-
bullying in a school context [in Spanish]. Psychology,
Society, & Education 2015; 7:213–226.
34. Jime
´nez AE, Castillo VD, Cisternas LC. Validation of the
aggression among peers scale and virtual aggression Sub-
scale with Chilean students [in Spanish]. Revista Latinoa-
mericana de Ciencias Sociales, Nin
˜ez y Juventud 2012;
10:825–840.
35. Baek J, Bullock LM. Cyberbullying: a cross-cultural per-
spective. Emotional and Behavioural Difficulties 2014; 19:
226–238.
36. Barlett CP, Gentile DA, Anderson CA, et al. Cross-cultural
differences in cyberbullying behavior: a short-term longi-
tudinal study. Journal of Cross-Cultural Psychology 2014;
45:300–313.
37. Said-Hung E. Vulnerable young people and mobile parti-
cipation in Colombia: a study of the levels of citizen par-
VALIDATION OF THE ECIPQ FOR COLOMBIA 123
ticipation and appropriation among beneficiaries of social
programs [in Spanish]. Innovar 2014; 24:31–44.
38. Montero I, Leo
´n OG. A guide for naming research studies
in psychology. International Journal of Clinical and Health
Psychology 2007; 7:847–862.
39. Ortega-Ruiz R, Del Rey R, Casas JA. Assessing bullying
and cyberbullying: Spanish validation of EBIPQ and
ECIPQ [in Spanish]. Psicologı
´a Educativa 2016; 22:71–79.
40. Brislin RW. (1986) The wording and translation of research
instruments. In Lonner W, Berry J, eds. Field methods in
cross-cultural research. Beverly Hills, CA: Sage, pp. 137–
164.
41. Bryant FB, Satorra A. Principles and practice of scaled
difference chi-square testing. Structural Equation Model-
ing: A Multidisciplinary Journal 2012; 19:372–398.
42. Jo
¨reskog KG. On the estimation of polychoric correlations
and their asymptotic covariance matrix. Psychometrika
1994; 59:381–389.
43. Bentler PM, Bonett DG. Significance tests and goodness of
fit in the analysis of covariance structures. Psychological
Bulletin 1980; 88:588–606.
44. Satorra A, Bentler PM. A scaled difference chi-square test
statistic for moment structure analysis. Psychometrika
2001; 66:507–514.
45. Hu L, Bentler PM. Cutoff criteria for fit indexes in co-
variance structure analysis: conventional criteria versus
new alternatives. Structural Equation Modeling: A Multi-
disciplinary Journal 1999; 6:1–55.
46. Bentler PM. (2005) EQS structural equations program
manual. Encino, CA: Multivariate Software.
47. Elosua OP, Zumbo BD. Reliability coefficients for ordered
categorical response scales [in Spanish]. Psicothema 2008;
20:896–901.
48. Lorenzo-Seva U, Ferrando PJ. FACTOR: a computer pro-
gram to fit the exploratory factor analysis model. Behavioral
Research Methods, Instruments and Computers 2006;
38:88–91.
49. Byrne BM, Shavelson RJ, Muthe
´n B. Testing for the
equivalence of factor covariance and mean structures: the
issue of partial measurement invariance. Psychological
Bulletin 1989; 105:456–466.
50. Dimitrov DM. Testing for factorial invariance in the con-
text of construct validation. Measurement and Evaluation in
Counseling and Development 2010; 43:121–149.
51. Bollen KA. (1989) Structural equations with latent vari-
ables. New York: Wiley.
52. Thomas HJ, Connor JP, Scott JG. Integrating traditional
bullying and cyberbullying: challenges of definition and
measurement in adolescents, a review. Educational Psy-
chology Review 2015; 27:135–152.
53. International Telecommunication Union (ITU). (2015)
Measuring the information society report 2015. Geneve:
ITU.
54. Almansa-Martı
´nez A, Fonseca O, Castillo-Esparcia A.
Social networks and young people. Comparative study of
Facebook between Colombia and Spain. Comunicar 2013;
20:127–135.
55. Festl R, Scharkow M, Quandt T. The individual or the
group: a multilevel analysis of cyberbullying in school
classes. Human Communication Research 2015; 41:535–
556.
56. Arango FG, Bringue
´SX, Sa
´daba CC. Interactive generation
in Colombia: teenagers before the internet, the cell phone,
and the videogames [in Spanish]. Anagrama 2010; 17:45–
56.
57. Lila M, Musitu G, Buelga S. Colombian and Spanish
adolescents: differences, similarities and relationships
between family socialization, self-esteem and values [in
Spanish]. Revista Latinoamericana de Psicologı
´a 2000;
32:301–319.
58. Arnett JJ. Broad and narrow socialization: the family in the
context of a cultural theory. Journal of Marriage and the
Family 1995; 57:617–628.
59. Romera EM, Herrera-Lo
´pez M, Casas JA, et al. Multi-
dimensional social competence, motivation, and cyberbul-
lying: a cultural approach with Colombian and Spanish
adolescents. Journal of Cross-Cultural Psychology. In press.
DOI: 10.1177/0022022116687854.
60. Stone AA, Turkkan J, Bacharach CA, et al. (1999) The
science of self report: implications for research and prac-
tice. Mahwah, NJ: Lawrence Erlbawn Associates.
Address correspondence to:
Dr. Mauricio Herrera-Lo
´pez
University of Narin
˜o
Ciudad Universitaria Torobajo
Calle 18 N50-40
CP 520002 San Juan de Pasto (Narin
˜o)
Colombia
E-mail: mherrera@udenar.edu.co
Appendix
Appendix Table A1. European Cyberbullying Intervention Project Questionnaire for Colombia
Cyber-victimization
ECIPQ 1 Alguien me ha dicho groserı
´as o insultado por internet (e-mail, redes sociales, llamadas o sms)
[Someone said nasty things to me or called me names using texts or online messages]
ECIPQ 2 Alguien ha dicho a otros groserı
´as sobre mı
´usando internet o sms (mensajes de celular) [Someone
said nasty things about me to others either online or through text messages]
ECIPQ 3 Alguien me ha amenazado a trave
´s de internet o sms (mensajes de celular) [Someone threatened me
through texts or online messages]
(continued)
124 HERRERA-LO
´PEZ ET AL.
Appendix Table AT1. (Continued)
ECIPQ 4 Alguien ha pirateado mi cuenta de correo y ha sacado mi informacio
´n personal (por ejemplo, a
trave
´s de e-mail o red social) [Someone hacked into my account and stole personal information
(e.g., through e-mail or social networking accounts)]
ECIPQ 5 Alguien ha pirateado mi cuenta y se ha hecho pasar por mı
´(a trave
´s de las redes sociales o e-mail)
[Someone hacked into my account and pretended to be me (e.g., through instant messaging or
social networking accounts)]
ECIPQ 6 Alguien ha creado una cuenta falsa en internet para hacerse pasar por mı
´(facebook, twitter,
whatsapp, e-mail, otra) [Someone created a fake account, pretending to be me (e.g., on Facebook
or MSN)]
ECIPQ 7 Alguien ha colgado informacio
´n personal sobre mı
´en internet [Someone posted personal
information about me online]
ECIPQ 8 Alguien ha colgado videos o fotos comprometedoras mı
´as en internet [Someone posted
embarrassing videos or pictures of me online]
ECIPQ 9 Alguien ha retocado fotos mı
´as que yo habı
´a colgado en internet [Someone altered pictures or
videos of me that I had posted online]
ECIPQ 10 He sido sacado (excluido) o ignorado de una red social o de chat [I was excluded or ignored by
others in a social networking site or Internet chatroom]
ECIPQ 11 Alguien ha difundido chismes (rumores) sobre mı
´por internet [Someone spread rumors about me
on the Internet]
Cyber-aggression
ECIPQ 12 He dicho groserı
´as o insultado a alguien usando mensajes por internet o sms (mensajes por celular)
[I said nasty things to someone or called them names using texts or online messages]
ECIPQ 13 He dicho groserı
´as sobre alguien a otras personas en mensajes por internet o sms [I said nasty things
about someone to other people either online or through text messages]
ECIPQ 14 He amenazado a alguien por internet o a trave
´s de mensajes de celular (sms) [I threatened someone
through texts or online messages]
ECIPQ 15 He pirateado (hackeado) la cuenta de correo o perfil de alguien y he robado su informacio
´n personal
(e-mail, red social) [I hacked into someone’s account and stole personal information
(e.g., through e-mail or social networking accounts)]
ECIPQ 16 He pirateado la cuenta o perfil de alguien y me he hecho pasar por e
´l/ella a trave
´s del chat, mensajes
o correos en las redes sociales [I hacked into someone’s account and pretended to be them
(e.g., through instant messaging or social networking accounts)]
ECIPQ 17 He creado una cuenta falsa para hacerme pasar por otra persona (por ejemplo en facebook, twitter,
chat, instagram u otra) [I created a fake account, pretending to be someone else (e.g., on
Facebook or MSN)]
ECIPQ 18 He colgado informacio
´n personal sobre alguien en internet (por ejemplo en redes sociales) [I posted
personal information about someone online]
ECIPQ 19 He colgado videos o fotos comprometedoras de alguien en internet [I posted embarrassing videos or
pictures of someone online]
ECIPQ 20 He retocado fotos o videos de alguien, que estaban colgados en internet [I altered pictures or videos
of another person that had been posted online]
ECIPQ 21 He excluido (sacado) o ignorado a alguien de una red social o de chat [I excluded or ignored
someone in a social networking site or Internet chatroom]
ECIPQ 22 He difundido rumores (chismes) sobre otras personas por internet [I spread rumors about someone
on the Internet]
ECIPQ, European Cyberbullying Intervention Project Questionnaire.
VALIDATION OF THE ECIPQ FOR COLOMBIA 125
