Content uploaded by Regina J J M van den Eijnden
Author content
All content in this area was uploaded by Regina J J M van den Eijnden on Apr 04, 2016
Content may be subject to copyright.
Full length article
The Social Media Disorder Scale: Validity and psychometric properties
Regina J.J.M. van den Eijnden
a
,
*
, Jeroen S. Lemmens
b
, Patti M. Valkenburg
c
a
Interdisciplinary Social Science, University of Utrecht, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands
b
Amsterdam School of Communication Research, University of Amsterdam, Nieuwe Achtergracht 166, 1001 NG, Amsterdam, The Netherlands
c
Amsterdam School of Communication Research, University of Amsterdam, Spui 21, 1012 WX Amsterdam, The Netherlands
article info
Article history:
Received 9 December 2015
Received in revised form
10 March 2016
Accepted 11 March 2016
Keywords:
Social Media Disorder
Social media addiction
Problematic social media use
Pathological social media use
Social media use
Internet addiction
abstract
There is growing evidence that social media addiction is an evolving problem, particularly among ad-
olescents. However, the absence of an instrument measuring social media addiction hinders further
development of the research field. The present study, therefore, aimed to test the reliability and validity
of a short and easy to administer Social Media Disorder (SMD) Scale that contains a clear diagnostic cut-
off point to distinguish between disordered (i.e. addicted) and high-engaging non-disordered social
media users.
Three online surveys were conducted among a total of 2198 Dutch adolescents aged 10 to 17. The 9-
item scale showed solid structural validity, appropriate internal consistency, good convergent and cri-
terion validity, sufficient test-retest reliability, and satisfactory sensitivity and specificity. In sum, this
study generated evidence that the short 9-item scale is a psychometrically sound and valid instruments
to measure SMD.
©2016 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND
license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
1. Introduction
The research field of Internet addiction continues to suffer from
definition and measurement problems. The concept Internet
addiction, also referred to as compulsive (Meerkerk, Van den
Eijnden, Vermulst, &Garretsen, 2009) or problematic Internet use
(Caplan, 2010), is multi-dimensional by nature and can refer to
different forms of compulsive online behaviors. Individuals do not
seem to be addicted to the Internet itself, but rather to certain
online activities (Griffiths &Szabo, 2013). Some of these activities,
however, appear to elicit more compulsive tendencies than others.
Among adolescents, the age group that rapidly adopts new tech-
nologies and is expected to be most vulnerable to possible negative
influences of these new technologies (Valkenburg &Peter, 2011),
Internet addiction has most convincingly been linked to gaming
and to social media use (Rumpf, Meyer, Kreutzer, John &Meerkerk,
2011; Van Rooij, Schoenmakers, Van den Eijnden, &Van de Mheen,
2010). Although the latest version of the Diagnostic and Statistical
Manual of Mental Disorders (DSM-5) recognizes Internet gaming
disorder as a tentative disorder in the appendix of this manual
(APA, 2013), social media addiction still has no status in the DSM-5.
While the exclusion of social media addiction from the DSM-5
may give the impression that social media addiction is not a
legitimate mental disorder, there is a growing body of evidence
suggesting otherwise (Pantic, 2014; Ryan, Chester, Reece, &Xenos,
2014). Moreover, there is empirical evidence indicating that
compulsive social media use is a growing mental health problem,
particularly among adolescent smartphone users (Van Rooij &
Schoenmakers, 2013). However, the absence of a clear definition
and a measure for social media addiction hampers research on the
prevalence of this type of disordered behavior, thereby obstructing
vital next steps in the research field of social media addiction.
Therefore, the present study aims to develop and validate a new
instrument for measuring social media addiction ethat is, the
Social Media Disorder (SMD) Scale.
Currently, the research field of social media addiction largely
lags behind research on game addiction. Whereas research on
game addiction has a long history dating back before online games
were available (e.g., Shotton, 1989; Soper &Miller, 1983), the social
media addiction field is relatively young, with the first studies
appearing after 2010 (for a review, see Ryan et al., 2014). Further-
more, while there are several validated instruments for measuring
game addiction (e.g. Griffiths, 2005, Lemmens, Valkenburg, &Peter,
2009; Lemmens, Valkenburg, &Gentile, 2015; Van Rooij,
Schoenmakers, Van den Eijnden, Vermulst, &Van de Mheen,
*Corresponding author. Department of Interdisciplinary Social Science, Utrecht
University, P.O. Box 80140, 3508 TC Utrecht, The Netherlands.
E-mail addresses: R.J.J.M.vandenEijnden@uu.nl (R.J.J.M. van den Eijnden), j.s.
lemmens@uva.nl (J.S. Lemmens), p.m.valkenburg@uva.nl (P.M. Valkenburg).
Contents lists available at ScienceDirect
Computers in Human Behavior
journal homepage: www.elsevier.com/locate/comphumbeh
http://dx.doi.org/10.1016/j.chb.2016.03.038
0747-5632/©2016 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Computers in Human Behavior 61 (2016) 478e487
2010), no validated instruments exist measuring social media
addiction. Instead, the field of social media addiction is character-
ized by an abundance of measurement instruments tapping into
particular forms of compulsive social media use, such as Facebook
addiction (Ryan et al., 2014), addiction to social network sites
(Griffiths, Kuss, &Demetrovics, 2014), Twitter addiction (Saaid, Al-
Rashid, &Abdullah, 2014), and microblogging dependence (Wang,
Lee, &Hua, 2015).
The fragmentation in the social media research field, along with
the proliferation of measures targeting specific forms of social
media addiction, is problematic for two reasons. First, the social
media landscape is characterized by rapid changes, whereby
existing social media platforms are expanded with new interactive
functions or simply replaced by new platforms. Instruments tar-
geting specific forms of social media addiction may thus be
outdated easily. Second, existing measures tend to use slightly
different criteria for eor operationalization ofesocial media
addiction, thereby hampering the comparability of research data
and stimulating further fragmentation of the field. Hence, to
accomplish the necessary progress in the field of social media
addiction, it is vital to develop and validate a general measure of
social media addiction based on a solid set of existing diagnostic
criteria.
There is ample ground for the development of a general social
media addiction instrument, since social media platforms share
many characteristics such as facilitating social interaction, the
sharing of ideas, formation and maintenance of relationships and/
or interest groups, and development of one's presence, reputation,
and identity (Kietzmann, Hermkens, McCarthy, &Silvestre, 2011).
Moreover, the finding that excessive use of different person-based
and group-based social media applications is related to Internet
addiction (Kuss &Griffiths, 2012; Kuss, Van Rooij, Shorter, Griffiths,
&Van de Mheen, 2013; Van den Eijnden, Meerkerk, Vermulst,
Spijkerman &Engels, 2007; Van Rooij et al., 2010) justifies the
development of a general social media addiction instrument.
1.1. Development and validation of the Social Media Disorder
(SMD) scale
The basic theoretical assumption underlying the development
of the Social Media Disorder (SMD) scale in the current study is that
social media addiction and Internet Gaming Disorder (IGD; APA,
2013) are two forms of the same overarching construct Internet
addiction and should therefore be defined by the same set of
diagnostic criteria. As stated before, the Internet incorporates a
variety of potential activities, and some of these activities, such as
gaming and social media use, tend to elicit compulsive tendencies
in a subgroup of users. Therefore, the measurement of SMD should
correspond with the measurement of both Internet addiction and
IGD. Thus, the same set of diagnostic criteria should be used when
operationalizing these related concepts.
In recent years, the addiction literature has extensively reflected
on the existence of non-substance related or behavioral addictions,
such as Internet addiction. In the absence of DSM-criteria for
Internet addiction, most instruments were based on the DSM-IV
criteria for substance dependence and/or pathological gambling.
More specifically, most instruments developed to assess Internet
addiction included at least six of the DSM-IV criteria for patho-
logical gambling, namely preoccupation, tolerance, withdrawal,
relapse, mood modification, and external consequences (see Van
Rooij &Prause, 2014). These six criteria were also recognized as
the core elements of behavioral addictions (Brown, 1993; Griffiths,
1999; Marks, 1990) and used for the development of most game
addiction measures (King, Haagsma, Delfabbro, Gradisar, &
Griffiths, 2013).
Based on thorough consideration by a multidisciplinary expert
group (see Petry et al., 2014), the APA decided to include three
additional criteria when defining the criteria for the DSM-5 diag-
nosis of IGD, namely deception (e.g., Demetrovics et al., 2012;
Gentile et al., 2011), displacement (e.g., Huang, Wang, Qian,
Zhong, &Tao, 2007; Rehbein, Kleimann, &M€
ossle, 2010), and
conflict (e.g., Lemmens et al., 2009;Young, 1996). Moreover, several
authors in the field of IGD refer to relapse as persistence, to mood
modification as escape, and to external consequences as problems
(Lemmens et al., 2015; Petry et al., 2014).
According to the DSM-5 definition, someone is diagnosed with
having IGD if he or she meets five (or more) of the nine criteria for
IGD during a period of 12 months. Since SMD and IGD are regarded
as two specific forms of the overarching construct Internet addic-
tion, it is reasoned that the nine criteria for IGD, which is the first
internet-related disorder included in the DSM, can also be used to
define SMD. The development of a SMD scale will thus be based on
the DSM-5 diagnostic criteria for IGD and will include the same
nine diagnostic criteria.
As suggested before, the development and validation of a
theoretically grounded and well-defined instrument to measure
SMD is essential in order to prevent the use of a large variety of
slightly different measurement instruments that do not allow for
clear-cut off points, and may not be applicable to multiple types of
social media. Moreover, there is a vital need for utilizing actual
clinical criteria in order to differentiate between pathological (i.e.
addictive) and highly engaged social media users (Kuss et al., 2013).
Thus, the development and validation of an instrument that is using
a clear diagnostic cut-off point, as provided by DSM-5, is crucial for
the development of this research field because such an instrument
offers the opportunity to assess and monitor the prevalence of
social media addiction in the population. Since SMD can be ex-
pected to be particularly disturbing for the psychosocial develop-
ment of adolescents (Valkenburg &Peter, 2011), the SMD scale will
be tuned to adolescents.
1.2. The current study
The main aim of the present study was to develop and validate a
scale to measure Social Media Disorder (SMD). Since SMD and IGD
are conceptualized as meeting at least five of the nine DSM-5
criteria for IGD, this study builds on a previous study testing the
reliability and validity of a short (9-item) and a long (27-item) scale
to measure IGD (Lemmens et al., 2015). Lemmens et al. (2015)
showed that the short 9-item scale, with a dichotomous (yes/no)
response scale, provided a valid and reliable measure of IGD with
good diagnostic accuracy, even in comparison to the long 27-item
scale. Because of the important advantages of a short and easy to
administer measurement instrument, such as the possibility to
incorporate the scale into space-limited surveys, and in agreement
with the findings by Lemmens et al. (2015), the ultimate aim of the
present study was to develop and validate a short 9-item scale to
measure SMD.
Our starting point was the development of a 27-item SMD scale,
consisting of three items for each of the nine DSM-5 criteria (see
Appendix A). After testing the factor structure and factor loadings
of this 27-item scale, the nine items with the highest factor loading
per criterion were selected to constitute the short 9-item scale.
Next, the psychometric properties of the short SMD scale were
tested, and compared with some psychometric properties of the
27-item SMD scale. More specifically, we examined (1) the factor
R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487 479
structure and internal consistency, (2) construct validity, as indi-
cated by convergent validity, (3) criterion validity, (4) test-retest
reliability, and (5) sensitivity and specificity of the short SMD
scale. These psychometric properties were tested across three
different samples of Dutch adolescents to establish population
cross-validation. Finally, we established the prevalence of SMD in
the current three samples, and tested for group differences in
gender, age, and frequency of daily social media use between
disordered and non-disordered adolescents.
Construct validity is defined as the extent to which the scale
measured the intended construct. Construct validity of the SMD
scale was established by testing the strength of the relationships
between scores on the SMD scale and constructs to which it should
theoretically be related. An important aspect of construct validity is
convergent validity, referring to the relation between the SMD scale
and comparable constructs. In this study convergent validity was
tested by relating scores on the SMD scale to Compulsive Internet
Use, as measured by the Compulsive Internet Use Scale (CIUS;
Meerkerk et al., 2009), and to self-declared social media addiction.
In case of good convergent validity, we expect to find strong cor-
relations between scores on the SMD scale and scores on the CIUS
and on self-declared social media addiction.
Criterion validity is defined as the extent to which a measure is
related to an outcome. Criterion validity of the SMD scale was
examined by testing the relationship between scores on the SMD
scale and several psychosocial constructs that have previously
been related to compulsive Internet use and (specific forms of)
compulsive social media use: self-esteem (Mehdizadeh, 2010; Van
Rooij et al., 2015), depression (Caplan, 2007; Hong, Huang, Lin, &
Chiu, 2014; Koc &Gulyagci, 2013; Yen, Ko, Yen, Wu, &Yang,
2007), loneliness (Caplan, 2007; Odaci &Kalkan, 2010; Van Rooij
et al., 2015), attention deficit (Dalbudak &Evren, 2014; Van Rooij
et al., 2015; Weinstein &Lejoyeux, 2010; Yen et al., 2007), and
impulsivity (Dalbudak &Evren, 2014; Wu, Cheung, Ku, &Hung,
2013). We expect to find weak to moderate correlations between
scores on the SMD scale and these psychosocial constructs. Finally,
in line with previous studies on the relationship between
compulsive Internet use and daily time online (Meerkerk et al.,
2009), and between IGD and time spent gaming (Lemmens et al.,
2015; Van Rooij et al., 2012), we expect to find moderate associa-
tions between the SMD scale and frequency of daily social media
use.
2. Method
2.1. Sample and procedure
From November 2014 through April 2015, three online surveys
were conducted among a total of 2198 Dutch adolescents, who
were all recruited through Marketing Science Institute (MSI), an
international market research company located in the Netherlands.
In November 2014, the first online questionnaire was distributed
among 724 teenagers (54% girls) aged 10e
17 (M¼14.36, SD ¼2.11).
Respondents received credit points for participating that could later
be redeemed for prizes. A second online survey was distributed two
months later among a sample of 873 adolescents, aged 10e17,
(M¼14.28, SD ¼2.15, 48% girls), of whom 238 had also completed
the first questionnaire. Finally, a third online survey was conducted
among a new sample of 601 adolescents aged 10e17 (M¼14.05,
SD ¼2.18, 50% girls).
2.2. Measures
The first online survey included the 27-item SMD scale, as well
as validity measures; that is, Compulsive Internet Use, Self-declared
Social Media Addiction, Self-esteem, Depression, Attention Deficit,
Impulsivity, and the use of several social media applications. The
second online survey contained the short 9-item SMD scale and
also measured Depression, Attention Deficit, Impulsivity, and the
additional variable Loneliness, in order to further test construct
validity. The third survey contained the 27-item SMD scale and a
wider range of items regarding smartphone usage than the first
survey (e.g., WhatsApp).
2.2.1. Social Media Disorder
The SMD scale consisted of 27 items (see Appendix A). Three
items were created for each of the previously identified nine
criteria: Preoccupation,Tolerance,Withdrawal,Displacement,Escape,
Problems,Deception,Displacement, and Conflict.
2.2.2. Compulsive Internet Use
Compulsive Internet use was assessed in the first sample using
the 14-item Compulsive Internet Use Scale (Meerkerk et al., 2009).
Example items are: ‘How often do you feel restless, frustrated, or
irritated when you cannot use the Internet?’and ‘How often do you
find it difficult to stop using the Internet when you are online?’
Items were assessed on a 5-point scale, ranging from (1) never to (5)
very often. Cronbach's alpha was 0.93 (M¼2.28, SD ¼0.78).
2.2.3. Self-declared Social Media Addiction
Respondents were asked: “To what extent do you feel addicted
to social media?”Answers to this question were given on a 5-point
scale ranging from (1) not at all addicted to (5) strongly addicted.
2.2.4. Self-esteem
The degree of self-esteem was measured using the six-item self-
esteem scale (Rosenberg, Schooler, &Schoenbach, 1989). This
measure implies feelings of self-acceptance, self-respect and
generally positive self-evaluation. Sample items are: “I am able to
do things at least as well as other people”and “I feel that I don't
have much to be proud of”(reverse coded). Response categories
ranged from 1 (totally disagree)to5(totally agree). The items were
averaged to create the scale scores. Cronbach's alpha for this scale
was 0.84 (M¼3.78, SD ¼0.73).
2.2.5. Depression
Depression was assessed using the 6-item Kutcher Adolescent
Depression Scale (LeBlanc, Almudevar, Brooks, &Kutcher, 2002).
Respondents were asked whether items were applicable to them
on a 5-point scale, ranging from never (1) to very often (5). Example
items are “I feel there is little hope for the future”and “I feel un-
happy and depressed”. Items were averaged to create scale scores.
Cronbach's alpha was 0.86 (M¼2.58, SD ¼0.84) in the first sample,
and 0.87 (M¼2.51, SD ¼0.85) in the second sample.
2.2.6. Attention deficit
The extent to which respondents displayed symptoms of
Attention Deficit was assessed by adapting nine items from the
DSM-IV checklist for ADHD that focused on attention deficit, or
inattention (APA, 2000). As proposed by Kessler et al. (2005), re-
spondents were asked to indicate how often nine situations were
applicable to them on a 5-point scale ranging from never (1) to very
often (5). Example items are “I am easily distracted”and “Ihave
difficulties organizing tasks”. Items were averaged to create scale
scores. Cronbach's alpha was 0.89 in the first sample (M¼2.59,
SD ¼0.74). In the second sample, Cronbach's alpha was 0.88
(M¼2.58, SD ¼0.71).
2.2.7. Impulsivity
The extent to which respondents displayed symptoms of
R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487480
impulsivity was assessed using six items adapted from the DSM-IV
checklist for ADHD that focused on impulsivity (APA, 2000).
Example items are “I have difficulty awaiting my turn”and “I
interrupt or intrude on others”. Respondents were asked to indicate
how often these six situations were applicable to them on a 5-point
scale ranging from never (1) to very often (5). Items were averaged
to create scale scores. Cronbach's alpha was 0.84 (M¼2.16,
SD ¼0.67) in the first sample. In the second sample, Cronbach's
alpha was 0.80 (M¼2.24, SD ¼0.72).
2.2.8. Loneliness
Feelings of loneliness were assessed with the 10-item Loneliness
Scale developed by Russell, Peplau, and Cutrona (1980);thisscale
contained 5 positive and 5 negative items. Examples of items are “I
feel completely alone”,“I have nobody to talk to”, and “there are
people who really understand me.”Negative items were recoded
before summing the 10 items into a scale.The internal consistency of
the scale was high; Cronbach's alpha was 0.90 (M¼2.18, SD ¼0.99).
2.2.9. Frequency of daily social media use
The frequency of daily social media use was measured by pre-
senting a list of the fifteen most popular social media. Respondent
were asked to indicate how often they used these social media on a
daily basis. Answer options were: (0) never (1) less than once a day
(2) 1e2 times (3) 3e5 times (4) 6e10 times (5) 11e20 times (6) 21e40
times (7) more than 40 times a day. Finally, we also asked for each
type of social media platform or app how often respondents posted
something, using the same 7-point scale.
2.3. Strategy of analyses
First of all, we tested whether the 27 items of the SMD scale,
consisting of three items for each of the nine DSM-5 criteria, can
be accounted for by one higher-order factor: social media disorder).
This factor structure was tested in the two independent samples,
the first and third one. We used structural equation modeling
(SEM) with weighted least squares estimators to test these second-
order factor models using CFA in MPlus (Asparouhov &Muth
en,
2009). Although maximum likelihood is the most common esti-
mation method in CFA, this method assumes that observed vari-
ables are continuous and normally distributed in the population
(Lubke &Muth
en, 2004). Because this assumption was not met
with our skewed distribution of SMD and ordinal levels of mea-
surement, a weighted least squares approach was applied to our
data, allowing any combination of dichotomous, ordered categori-
cal, or continuous observed variables (Flora &Curran, 2004).
Although researchers sometimes correlate error terms on the basis
of theoretically overlapping indicators in an effort to improve
model fit, this should be avoided if possible, since it means that
there is some other issue that is not specified within the model that
is causing the covariation (Hooper, Coughlan, &Mullen, 2008).
Therefore, the error terms associated with each observed item are
uncorrelated (Byrne, 2001).
The goodness of fit was evaluated using the chi-square value,
the Comparative Fit Index (CFI), the Root Mean Square Error of
Approximation (RMSEA), and its 90% confidence interval (CI).
Particularly when dealing with large samples, the chi square test is
not a good indicator of fit, and the CFI and RMSEA indices are
considered informative fit criteria in SEM (Byrne, 2001). A good fit
is expressed by a CFI greater than 0.95 and a RMSEA value less than
0.08 (Byrne, 2001; Hu &Bentler, 1999; Yu &Muth
en, 2002). In
addition, the internal consistency of the 27-item scale was calcu-
lated by means of Cronbach's alpha.
Next, a short 9-item version of the SMD scale was developed
that encompasses all DSM-5 criteria by selecting the highest
loading items from each criterion. The standardized item-loadings
from sample 1 were used to select a set of nine items with the
highest overall loadings from each of the nine first-order factors.
This short version of the scale was then tested as a first-order
structural model using Mean- and Variance-adjusted Weighted
Least Square (WLSMV) estimators in Mplus. Internal consistency of
the 9-item scale was calculated in all three samples by means of
Cronbach's alpha.
We also investigated the population cross-validity of the one-
dimensional structure of the short 9-item scale. More specifically,
we tested whether the hypothesized one-dimensional structure of
the short SMD scale, which was found in the first sample, was also
found in the second and third sample. Population cross-validity is
satisfactory when the results found in one sample of a population
can also be found in another independent sample drawn from the
same population (e.g., Raju, Bilgic, Edwards, &Fleer, 1997).
After testing the factor structure and internal consistency of the
long 27-item and short 9-item SMD scale (first aim), we examined
the construct validity of these scales (second aim). More specif-
ically, we assessed convergent validity, which can be established if
two similar constructs correspond with one another. To assess
convergent validity, respondents’sum scores on the SMD scale
were correlated with compulsive Internet use, and self-declared
social media addiction. Next, we determined criterion validity
(third aim), that is, the extent to which a measure is related to an
outcome that it theoretically should be related to. We assessed
criterion validity by correlating the scores on the SMD scale with
self-esteem, depression, loneliness, attention deficit, and impul-
sivity. The following criteria were used to classify magnitude of
correlations: small, r¼.1e.29; medium, r¼.3e.49; large, r¼.5e1.0
(Cohen, 1960).
The fourth aim was to calculate the test-retest reliability of the
short 9-item SMD scale. We investigated the test-retest reliability
by computing Pearson correlations between scores on the short
SMD scales among the 238 adolescents who participated in the first
and second online survey. In addition, the intra-class correlation
coefficient (ICC) was established, using a two factor mixed effects
model and type consistency (McGraw &Wong,1996). The fifth aim
was to determine the sensitivity and specificity of the nine items of
the short SMD scale. However, before doing so, we examined the
prevalence of SMD in the three samples as indicated by the five-or-
more cut-off point of the short SMD scale, and we tested for group
differences in gender, age, and frequency of daily social media use
between disordered and non-disordered adolescents. Next, sensi-
tivity was demonstrated by the proportion of disordered social
media users who answered ‘yes’on an indicator of SMD, whereas
specificity was indicated by the proportion of non-disordered users
who reported ‘no’on an indicator of SMD.
3. Results
3.1. Social media use
The reported results on social media use are derived from the
combined samples 1 and 3 (N¼1325) unless otherwise specified. A
small group (Sample 1: 6.6%, n¼88; Sample 2: 10,3%, n¼90) re-
ported not using any form of social media and was excluded from
analyses. Out of all 1237 social media users, 92.2% (n¼1140) re-
ported owning a smartphone and using it for social media. The
most popular social media platforms and apps are displayed in
Table 1.
R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487 481
3.2. The dimensional structure of the SMD scale
The 27-item scale was included in the first sample (N¼724),
M¼5.65, SD ¼5.5 and in the third sample (N¼601), M¼5.65,
SD ¼6.17. For analyses, all yes-answers were summed (range 0e27).
The dimensional structure of the 27-item SMD scale (3 items per
criterion) was tested using a second-order factor model. This
resulted in an acceptable model fit,
c
2
(288, n¼724) ¼672.424,
p<0.001, CFI ¼0.963, RMSEA ¼0.043 (90% CI: 0.039e0.047) in the
first sample. Similarly, in the third sample (n¼601), the same
model also showed an acceptable model fit,
c
2
(288,
n¼601) ¼570.681, p<0.001, CFI ¼0.973, RMSEA ¼0.040 (90% CI:
0.036e0.045). Moreover, the 27-item SMD scale showed good in-
ternal consistency with a Cronbach's alpha of 0.90 in the first
sample and 0.92 in the third sample. Table 2 shows the factor
loadings and percentages of affirmative answers for all 27 items in
samples 1 and 3.
3.3. Constructing a short SMD scale and testing population cross-
validity
In order to facilitate incorporation of the SMD scale into space-
limited surveys, and assess the prevalence of SMD among adoles-
cents, an important aim of this study was to investigate whether a
9-item model would provide an equal or even better description of
the data. In sample 1, the unconstrained first-order structural 9-
item model using Mean- and Variance-adjusted Weighted Least
Square (WLSMV) estimators yielded a good fit,
c
2
(27,
n¼724) ¼24.846, p¼0.58, CFI ¼1.000, RMSEA ¼0.000 (90% CI:
0.000e0.026). This short SMD scale was strongly correlated with
the 27-item SMD scale (r¼0.89, p<0.001) and showed good
reliability with a Cronbach's alpha of 0.81 (M¼1.22, SD ¼1.87).
Items for the short 9-item SMD scale are displayed in Table 3. The
total time to complete the short 9-item SMD scale was about 45 s,
compared to about 2 min and 15 s for completing the 27-item scale.
In a next step, we examined the population cross-validity by
testing whether the one-dimensional structure of the short SMD
scale that was found in the first sample, could also be found in
the second and third sample. Again, the unconstrained first-order
structural 9-item model yielded a good fit,
c
2
(27,
n¼873) ¼62.852, p¼0.001, CFI ¼0.997, RMSEA ¼0.041 (90% CI:
0.028e0.055). Furthermore, the 9-item scale showed adequate
reliability with a Cronbach's alpha of 0.76 (M¼1.94, SD ¼2.11).
Finally, in sample 3, the unconstrained first-order structural 9-item
model also yielded a good fit,
c
2
(27, n¼601) 54.129, p¼0.002,
CFI ¼0.989, RMSEA ¼0.041 (90% CI: 0.025e0.057). In this sample,
the 9-item scale also showed a strong correlation with the 27-item
scale (r¼0.94, p<0.001) and showed good reliability with a
Cronbach's alpha of 0.82 (M¼1.52, SD ¼2.11).
3.4. Convergent and criterion validity of the SMD scales
In order to establish the convergent validity, respondents’mean
scores on the long and short SMD scales were correlated with
compulsive Internet use and self-declared social media addiction.
Next, to assess the criterion validity, the SMD scales were correlated
with dissimilar but related constructs, i.e. depression, self-esteem,
loneliness, attention deficit, impulsivity, and frequency of daily
social media use. As Table 4 shows, all correlations were significant
at least at p<0.001 in the expected directions. The long (27-item)
and short (9-item) versions of the SMD scale both showed large
positive correlations with compulsive Internet use (r >0.50) and
medium to large correlations with self-declared social media
addiction, (r >0.48), indicating satisfactory convergent validity.
With regard to criterion validity, the long and short SMD scales
showed medium positive correlations with depression, attention
deficit, and frequency of daily social media use and posts, and weak
to moderate positive associations with loneliness and impulsivity
(see Table 4). Finally, a small negative correlation with self-esteem
was found. The correlations between the SMD scales and these
related constructs indicated good criterion validity. Overall, the
strength of the correlations between the SMD scales and these
similar and related constructs was somewhat lower for the 9-item
scale than for the 27-item scale, but the 9-item scale still demon-
strated satisfactory convergent and criterion validity.
Table 1
The most popular social media (N ¼1325).
Total users Users on smartphone
a
Daily posts (1 post) Daily posts (>10 posts)
Facebook 83% 68% 46% 1%
WhatsApp
b
82% 82% 82% 32%
Instagram 54% 51% 41% 1%
YouTube 53% 43% 33% 1%
Twitter 34% 26% 19% 2%
Note:
a
Proportion of the total sample (N¼1325);
b
WhatsApp was measured only in survey 3 (N¼601).
Table 2
Affirmative answers and confirmatory factor loadings of SMD items.
# Criterion Sample 1 (n¼724) Sample 3 (n¼601)
% yes Loadings (
b
) % yes Loadings (
b
)
1 Preoccupation1 44 0.686 44 0.676
2 Preoccupation2 12 0.784 13 0.860
3 Preoccupation3 30 0.554 32 0.631
4 Tolerance1 39 0.713 35 0.829
5 Tolerance2 32 0.742 30 0.833
6 Tolerance3 09 0.902 10 0.938
7 Withdrawal1 22 0.781 24 0.833
8 Withdrawal2 16 0.886 21 0.894
9 Withdrawal3 13 0.946 14 0.963
10 Persistence1 18 0.854 18 0.876
11 Persistence2 16 0.975 17 0.896
12 Persistence3 16 0.908 15 0.926
13 Displacement1 29 0.773 33 0.769
14 Displacement 2 18 0.919 22 0.844
15 Displacement 3 13 0.903 16 0.873
16 Problems1 27 0.647 30 0.684
17 Problems2 35 0.601 34 0.666
18 Problems3 09 0.889 09 0.858
19 Deception1 14 0.814 16 0.928
20 Deception2 13 0.760 13 0.947
21 Deception3 20 0.760 20 0.936
22 Escape1 28 0.919 27 0.905
23 Escape2 23 0.954 24 0.819
24 Escape3 20 0.965 19 0.834
25 Conflict1 08 0.814 11 0.874
26 Conflict2 08 0.877 11 0.825
27 Conflict3 06 0.842 05 0.809
Note: Item descriptions are found in Appendix A.
R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487482
3.5. Test-retest reliability of the 9-item SMD scale
Test-retest reliability of the 9-item short SMD scale was assessed
among the 238 adolescents who participated in both the first and
the second online survey (with an interval of 2 months between
these two surveys). A moderate degree of reliability was found
between the first and second SMD scales. The Pearson correlation
between both scales was 0.50, p <0.001. The averaged measure ICC,
using an absolute agreement definition, was.663 (95%
CI:.565e.739), and the mean variation between the measures of
SMD was 0.47. An averaged measure ICC of 0.60 or higher indicates
satisfactory stability (Landis &Koch, 1977).
3.6. Prevalence of Social Media Disorder
The 9-item SMD scale was used to assess the prevalence of
disordered social media use among teenagers. In accordance with
the cut-off point for IGD in the DSM-5, at least five or more (out of
nine) criteria must be met for a formal diagnosis of ‘disordered
social media user’. Among the first sample (N¼724), we found that
53 teenagers met five or more of the criteria (7.3%). In the second
sample (N¼873), 101 adolescents (11.6%) met the cut-off point for
disordered use of social media. In the third sample (N¼601), 62
teenagers (10.3%) could be viewed as disordered social media users.
Also, we examined whether disordered social media users
differed from non-disordered users with regard to gender, age, and
frequency of daily social media use. Chi-square tests for the first
sample indicated that there were more disordered boys (n¼34,
10.2%) than disordered girls (n¼19, 4.9%),
c
2
(1, 724) ¼7.471,
p¼0.006. The second and third sample, however, did not replicate
this gender difference: in the second sample, the number of boys
among disordered social media users (n¼45, 9.9%) did not differ
from the number of disordered girls (n¼56, 13.3%),
c
2
(1,
873) ¼2.462, p¼0.117. Similarly, in the third sample no differences
between boys (n¼26, 8.7%) and girls (n¼36, 12.0%),
c
2
(1,
601) ¼1.762, p¼0.148, were found.
With regard to age, no differences were found between disor-
dered and non-disordered users (sample 1, t(1,722) ¼1.40,
p¼0.16; sample 2, t(1,781) ¼0.60, p¼0.55; sample 3, t
(1,599) ¼0.30, p¼0.77). Table 5 illustrates the differences in use
of specific social media applications between disordered and non-
disordered social media users. In the first sample, all social media
are used more often among disordered users. In the third sample,
significant differences were only demonstrated for active use of
Facebook, Instagram, and Whatsapp.
3.7. Sensitivity and specificity of the 9-item SMD scale
Finally, each of the nine indicators of the SMD scale was
examined for their sensitivity and specificity. Sensitivity of the
short scale items was demonstrated by the proportion of disor-
dered social media users from each sample (n's ¼53, 101, 63) who
answered positively on an item, whereas specificity was indicated
by the proportion of negative responses on a scale item by non-
disordered gamers from each sample. Ideally, both sensitivity and
specificity of an item should be high in order to discriminate false
positives and false negatives (Glaros &Kline, 1988). As Table 6
shows, the nine items show adequate sensitivity and high speci-
ficity. The diagnostic accuracy, as indicated by the proportion of all
true positives (indicating sensitivity) and true negatives (indicating
specificity), was highly comparable across samples. The sensitivity
of Problems and Conflict were the lowest of all items across samples,
indicating that between 47% and 62% of all disordered social media
users had experienced serious problems as a result of their
compulsive social media use, and that between 50% and 61% of the
disordered users had experienced conflicts with friends, family or
partners because of their social media use. Conversely, the speci-
ficity of Problems and Conflict was high across samples (ranging
between 0.92 and 0.97) indicating that between 3% and 8% of the
social media users who had experienced problems and conflicts
were not among the disordered gamers.
Table 3
The 9-item SMD scale.
Criterion During the past year, have you …
Preoccupation …regularly found that you can't think of anything else but the moment that you will be able to use social media again?
Tolerance …regularly felt dissatisfied because you wanted to spend more time on social media?
Withdrawal …often felt bad when you could not use social media?
Persistence …tried to spend less time on social media, but failed?
Displacement …regularly neglected other activities (e.g. hobbies, sport) because you wanted to use social media?
Problem …regularly had arguments with others because of your social media use?
Deception …regularly lied to your parents or friends about the amount of time you spend on social media?
Escape …often used social media to escape from negative feelings?
Conflict …had serious conflict with your parents, brother(s) or sister(s) because of your social media use?
Table 4
Correlations Between the 9- and 27-item SMD scales and Validation Constructs.
Sample 1 (N ¼724) Sample 2 (N ¼873) Sample 3 (N ¼601)
SMD 27 SMD 9 SMD 9 SMD 27 SMD 9
CIUS 0.57 0.51 eee
Self-declared SMD 0.60 0.48
Self-Esteem 0.19 0.19 eee
Depression 0.37 0.29 0.29 ee
Loneliness ee0.24 ee
Attention Deficit 0.36 0.33 0.26 ee
Impulsivity 0.30 0.30 0.27 ee
Frequency Daily Use 0.35 0.25 e0.25 0.20
Frequency Daily Posts 0.34 0.27 e0.28 0.24
Note: All correlations are significant at p<0.001.
R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487 483
4. Discussion
There is a growing body of evidence suggesting that social
media disorder (i.e. addiction) is an emerging mental problem,
particularly among adolescents (Pantic, 2014; Ryan et al., 2014).
However, the absence of a measurement tool for SMD hampers
further development of the research field. Particularly, there is a
strong need for an assessment instrument that can distinguish
between disordered (i.e. addictive) and highly engaged non-
disordered social media users. Therefore, the present study, con-
sisting of three online surveys among adolescents aged 10 to 17,
aimed to test the reliability and validity of a short and easy to
administer SMD Scale that contains a clear diagnostic cut-off point.
In the absence of specific diagnostic criteria for SMD, the
development of our measurement tool was based on the assump-
tion that SMD and IGD are two forms of the same overarching
construct Internet Addiction, and should thus be defined by the
same set of diagnostic criteria. Therefore, the development of the
SMD scale was based on the nine DSM-5 criteria for IGD (APA,
2013). First, a 27-item dichotomous scale was developed assess-
ing the nine DSM-5 criteria, i.e. preoccupation, tolerance, with-
drawal, persistence, escape, problems, deception, displacement,
and conflict, with three items per criterion. Next, we examined
whether a short 9-item scale, consisting of the nine items with the
highest factor loading per criterion, would provide an equally valid
and reliable measurement tool.
This study generated evidence that the 9-item scale is a psy-
chometrically sound and valid instrument to measure SMD, and
just as valid as the 27-item version. Confirmatory factor analysis
showed good model fits, indicating solid structural validity. The 9-
item scale also showed appropriate internal consistency, sufficient
test-retest reliability, and good convergent and criterion validity.
Moreover, the nine items generally showed adequate sensitivity
and good specificity. The prevalence of SMD, determined on basis of
the diagnostic cut-off point of the 9-item scale, ranged from 7.3% to
11.6% in the 3 online samples.
Convergent validity was determined by the strength of corre-
lations between SMD and similar constructs. As expected, corre-
lations between scores on the SMD scales and scores on the
Compulsive Internet Use Scale (CIUS) were strong. Also, the SMD
scales showed strong correlations with self-reported social media
addiction, although the strength of this correlation was somewhat
lower for the 9-item scale than for the 27-item scale. The strength
of the association between the 9-item scale and self-declared social
media addiction, however, is substantial and similar to the strength
of the previously found association between scores on the CIUS and
self-reported Internet addiction (Meerkerk et al., 2009).
Criterion validity was determined on basis of the strength of
correlations with psychosocial constructs that were previously
related to compulsive Internet use and/or social media use. In line
with our expectations, the SMD scales showed moderate relation-
ships in the expected direction with depression, attention deficit
and frequency of daily social media use, weak to moderate asso-
ciations with loneliness and impulsivity. However, only a weak
negative association was found with self-esteem.
The relatively stronger associations found for depression and
attention deficit coincide with findings of a recent study indicating
that depressive mood and hyperactivity-inattention are more
strongly related to SMD than to IGD (Van Rooij et al., 2015). The
relatively strong link between SMD and attention deficit corrobo-
rates the public concern that the use of social media, particularly
when used via smartphones (e.g., Whatsapp), is distracting ado-
lescents’attention from their everyday activities and obligations.
A recent review indeed suggests that higher levels of media
Table 5
Mean frequency of daily social media use
a
of disordered and non-disordered users.
Sample 1 (N ¼724) Sample 3 (N ¼601)
Non-disordered Disordered Non-disordered Disordered
Facebook use 3.20 (1.44) 3.70 (1.53)
*
3.26 (1.33) 3.57 (1.64)
Facebook posts 1.43 (0.67) 2.13 (1.15)
*
1.64 (0.90) 2.08 (0.99)
*
FB messenger use ee2.44 (1.18) 3.03 (1.55)
*
FB messenger posts ee1.88 (1.08) 2.60 (1.42)
*
Instagram use 3.44 (1.52) 4.06 (1.63)
*
3.54 (1.37) 3.78 (1.71)
Instagram posts 1.60 (0.84) 2.31 (1.53)
*
1.75 (0.91) 2.22 (0.87)
*
YouTube use 1.21 (0.70) 1.57 (1.07)
*
3.30 (1.33) 3.37 (1.33)
YouTube posts 1.21 (0.69) 1.57 (1.07)
*
1.58 (0.99) 1.86 (1.04)
Twitter use 1.60 (1.17) 2.70 (2.05)
*
2.41 (1.38) 2.88 (1,80)
Twitter posts 1.59 (1.17) 2.70 (2.05)
*
1.73 (0.99) 1.88 (1.00)
WhatsApp posts ee4.13 (1.58) 5.08 (1.71)
*
Note: (SD),
*
significant t-test differences at least p <0.01;
a
Answer options were: (0) never, (1) less than once a day, (2) 1e2 times, (3) 3e5 times, (4) 6e10 times, (5) 11e20
times, (6) 21e40 times, and (7) more than 40 times a day.
Table 6
Sensitivity and specificity of the nine criteria for Social Media Disorder.
Disordered users sample 1 (n¼53) Disordered users sample 2 (n¼101) Disordered users sample 3 (n¼62)
Sens eSpec Sens eSpec Sens eSpec
Preoccupation 0.74e0.92 0.79e0.87 0.77e0.94
Tolerance 0.74e0.97 0.66e0.93 0.66e0.96
Withdrawal 0.87e0.94 0.78e0.85 0.81e0.93
Persistence 0.81e0.88 0.69e0.77 0.69e0.83
Displacement 0.62e0.97 0.76e0.83 0.73e0.84
Problems 0.62e0.92 0.55e0.93 0.47e0.95
Deception 0.77e0.91 0.66e0.89 0.71e0.90
Escape 0.79e0.82 0.66e0.83 0.71e0.82
Conflict 0.59e0.97 0.50e0.95 0.61e0.94
R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487484
multitasking, i.e. the use of media while engaging in non-media
activities, such as completing homework and engaging in face-to-
face interactions, is related to deficits in cognitive control, in
particular to the ability to sustain attention (Van der Schuur,
Baumgartner, Sumter, &Valkenburg, 2015). The relatively high
correlation between SMD and attention deficit thus provides some
evidence for the scattered attention hypothesis (e.g., Ophira, Nass,
&Wagner, 2009) which states that when people frequently
engage in media multitasking, they become accustomed to con-
stant switching between activities and eventually lose their ability
to focus on a single activity (Van der Schuur et al., 2015; Wallis,
2006, 2010). However, because of the cross-sectional nature of
the present findings, no causal inferences can be made. Future
longitudinal and experimental research is warranted to establish
causality between SMD and attention deficit in adolescents.
Most of the nine items of the short SMD scale showed good
sensitivity and specificity. The items measuring Problems and
Conflict, however, showed a lower sensitivity in comparison to the
other items of the SMD scale, as well as in comparison to Problems
and Conflict items of the IGD-scale (Lemmens et al., 2015). Experi-
encing conflict with others about the time spent on social media
use may have less external validity than conflicts about time spent
gaming. Social media use is more easily stopped or combined with
other activities, thereby causing fewer problems as a result of
compulsive social media use, in comparison to compulsive gaming.
Also, having to quit gaming may be experienced as more frustrating
by disordered adolescents than having to quit social media use.
Consequently, IGD may induce more conflict and arguing with
family members than SMD. Previous research indeed showed that
disordered gamers display more physical aggression (Lemmens,
Valkenburg, &Peter, 2011) than non-disordered gamers. Future
research should address the aptness of the nine DSM-5 criteria for
measuring SMD, and examine whether Problems and Conflict are
indeed core features of SMD, as was assumed in the present study.
The findings of the third survey suggest that some types of social
media use may elicit a higher risk than others, and that disordered
users differ from non-disordered users particularly in the number
of posts that they place on Facebook, Instagram and Whatsapp.
However, these results are somewhat inconsistent with the find-
ings of the first survey suggesting that both passive and active use
of social media is related to SMD. Future research should address
the strength of the relationships between different types of social
media use and SMD in more detail.
Some limitations of the present research warrant discussion.
First, the nine DSM-5 criteria defined for IGD were translated to
SMD. It should be noted, however, that these nine criteria for IGD
are still subject to discussion (e.g. Griffiths et al., 2015; Kardefelt-
Winther et al., 2014). Consequently, using the same nine DSM-5
criteria of IGD to measure SMD may yield similar conceptual de-
bates. For instance, the notion that the criterion of Deception, i.e.
lying about the time spend on social media, is socially or culturally
subjective and also depend on the people close to the gamer
(Kardefelt-Winther et al., 2014), also applies to the current
conceptualization of social media disorder. In addition, some of the
proposed IGD criteria may be less relevant in the context of social
media use. As suggested earlier, Conflict and Problems may be less
appropriate criteria to measure SMD, as compared to IGD, and thus
strongly require further investigation.
Despite these conceptual shortcomings, we followed the prag-
matic approach of developing and validating this short and easy to
administer tool to measure SMD, which enables the investigation of
trends and developments in the prevalence of SMD during this
period of rapid changes in the social media landscape. This study,
however, is regarded a first research step, and an important next
step would be to investigate the correctness of the nine DSM-5
criteria as the core features of SMD, as well as to test whether the
items covered in the short SMD scale indeed are the most suitable
ones for diagnosing SMD in both clinical and non-clinical samples.
It would be interesting, for instance, to test the extent to which self-
declared social media addicts identify with the items of both the
short 9-item and the long 27-item SMD scale to gain more insight
into the actual significance of the nine DSM-5 criteria for deter-
mining SMD. After these research steps have been taken, this in-
strument will facilitate the investigation of psychological processes
(motivational, affective, cognitive, interpersonal, and social) sus-
taining the dysfunctional involvement in social media use (Billieux,
Schimmenti, Khazaal, Maurage, &Heeren, 2015; Dudley, Kuyken, &
Padesky, 2011), and will thereby contribute substantially to un-
derstanding Social Media Disorder.
Appendix A
27 items for the Social Media Disorder Scale.
Preoccupation
During the past year, have you …
…often found it difficult not to look at messages on social media when you were doing something else (e.g. school work)?
…regularly found that you can't think of anything else but the moment that you will be able to use social media again?*
…often sat waiting until something happens on social media again?
Tolerance
During the past year, have you …
…felt the need to use social media more and more often?
…felt the need to check messages on social media more and more often?
…regularly felt dissatisfied because you wanted to spend more time on social media?*
Withdrawal
During the past year, have you …
…often felt tense or restless if you weren't able to look at your messages on social media?
…regularly felt angry or frustrated if you weren't able to use social media?
…often felt bad when you could not use social media?*
Persistence
During the past year, have you …
…tried to reduce your use of social media, but failed?
…tried to spend less time on social media, but failed?*
…been unable to stop using social media, even though others told you that you really should?
Escape
During the past year, have you …
…regularly used social media to take your mind off your problems?
(continued on next page)
R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487 485
References
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental
disorders (4th ed.). Washington, DC: American Psychiatric Pub. Association. text
revision.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental
disorders (5th ed.). Washington, DC: American Psychiatric Association. text
revision.
Asparouhov, T., & Muth
en, B. (2009). Exploratory structural equation modeling.
Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 397e438.
Billieux, J., Schimmenti, A., Khazaal, Y., Maurage, P., & Heeren, A. (2015). Are we over
pathologizing everyday life? A tenable blueprint for behavioral addiction
research. Journal of behavioral addictions, 4(3), 119e123.
Brown, R. I. F. (1993). Some contributions of the study of gambling to the study of
other addictions. In W. R. Eadington, & J. A. Cornelius (Eds.), Gambling behavior
and problem gambling (pp. 241e272). Reno, NV: University of Nevada.
Byrne, B. M. (2001). Structural equation modeling with AMOS: Basic concepts, appli-
cations and programming. Mahwah, NJ: Erlbaum.
Caplan, S. E. (2007). Relations among loneliness, social anxiety, and problematic
Internet use. CyberPsychology &Behavior, 10, 234e242.
Caplan, S. E. (2010). Theory and measurement of generalized problematic Internet
use: a two-step approach. Computers in Human Behavior, 26(5), 1089e1097.
Cohen, J. A. (1960). A coefficient of agreement for nominal scales. Educational and
Psychological Measurement, 20(1), 37e46.
Dalbudak, E., & Evren, C. (2014). The relationship of Internet addiction severity with
Attention Deficit Hyperactivity Disorder symptoms in Turkish University stu-
dents; impact of personality traits, depression and anxiety. Comprehensive
Psychiatry, 55(3), 497e503.
Demetrovics, Z., Urb
an, R., Nagygy€
orgy, K., Farkas, J., Griffiths, M. D., P
apay, O., et al.
(2012). The development of the problematic online gaming questionnaire
(POGQ). PLoS ONE, 7.http://dx.doi.org/10.1371/journal.pone.0036417.
Dudley, R., Kuyken, W., & Padesky, C. A. (2011). Disorder specific and trans-
diagnostic case conceptualisation. Clinical Psychology Review, 31(2), 213e224.
Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of
estimation for confirmatory factor analysis with ordinal data. Psychological
Methods, 9, 466e491. http://dx.doi.org/10.1037/1082-989X.9.4.466.
Gentile, D. A., Choo, H., Liau, A., Sim, T., Li, D., Fung, D., et al. (2011). Pathological
video game use among youth: a two-year longitudinal study. Pediatrics, 127,
319e329. http://dx.doi.org/10.1542/peds.2010-1353.
Glaros, A. G., & Kline, R. B. (1988). Understanding the accuracy of tests with cutting
scores: the sensitivity, specificity, and predictive value model. Journal of Clinical
Psychology, 44,1013e1023.
Griffiths, M. (1999). Internet addiction: fact or fiction? Psychologist, 12(5), 246e250.
Griffiths, M. (2005). A ‘components’model of addiction within a biopsychosocial
framework. Journal of Substance Use, 10,191e197.
Griffiths, M., Van Rooij, A. J., Kardefelt-Winther, D., Starcevic, V., Kir
aly, O.,
Pallesen, S., et al. (2016). Working towards an international consensus on
criteria for assessing Internet Gaming Disorder: a critical commentary on Petry,
et al (2014). Addiction, 111(1), 167e175 .
Griffiths, M. D., Kuss, D. J., & Demetrovics, Z. (2014). Social networking addiction: an
overview of preliminary findings. In Behavioral addictions. Criteria, evidence, and
treatment (pp. 119e141). New York: Elsevier.
Griffiths, M., & Szabo, A. (2013). Is excessive online usage a function of medium or
activity? an empirical pilot study. Journal of behavioral addictions, 3(1), 74e77.
Hong, F., Huang, D., Lin, H., & Chiu, S. (2014). Analysis of the psychological traits,
Facebook usage, and Facebook addiction model of Taiwanese university
students. Telematics and Informatics, 31,597e606.
Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling:
guidelines for determining model fit. Electronic Journal of Business Research
Methods, 6,53
e60.
Huang, Z., Wang, M., Qian, M., Zhong, J., & Tao, R. (2007). Chinese internet addiction
inventory: developing a measure of problematic internet use for Chinese col-
lege students. Cyberpsychology &Behavior, 10, 805e811. http://dx.doi.org/
10.1089/cpb.2007.9950.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure
analysis: conventional criteria versus new alternatives. Structural Equation
Modeling, 6,1e55. http://dx.doi.org/10.1080/10705519909540118.
Kardefelt-Winther, D. (2015). A critical account of DSM-5 criteria for internet
gaming disorder. Addiction Research &Theory, 23(2), 93e98.
Kessler, R. C., Adler, L., Ames, M., Demler, O., Faraone, S., Hiripi, E. V. A., et al. (2005).
The World Health Organization Adult ADHD Self-Report Scale (ASRS): a short
screening scale for use in the general population. Psychological Medicine, 35(02),
245e256.
Kietzmann, J. H., Hermkens, K., McCarthy, I. P., & Silvestre, B. S. (2011). Social media?
Get serious! Understanding the functional building blocks of social media.
Business Horizons, 54(3), 241e251.
King, D. L., Haagsma, M. C., Delfabbro, P. H., Gradisar, M., & Griffiths, M. D. (2013).
Toward a consensus definition of pathological video-gaming: a systematic re-
view of psychometric assessment tools. Clinical Psychology Review, 33(3),
331e342.
Koc, M., & Gulyagci, S. (2013). Facebook addiction among Turkish college students:
the role of psychological health, demographic, and usage characteristics.
CyberPsychology, Behavior, and Social Networking, 16,279e284.
Kuss, D. J., & Griffiths, M. D. (2012). Internet gaming addiction: a systematic review
of empirical research. International Journal of Mental Health and Addiction, 10(2),
278e296.
Kuss, D. J., Van Rooij, A. J., Shorter, G. W., Griffiths, M. D., & Van de Mheen, D. (2013).
Internet addiction in adolescents: prevalence and risk factors. Computers in
Human Behavior, 29(5), 1987e1996.
Landis, J., & Koch, G. (1977). The measurement of observer agreement for categorical
data. Biometrics, 33,159e174 .
LeBlanc, J. C., Almudevar, A., Brooks, S. J., & Kutcher, S. (2002). Screening for
adolescent depression: comparison of the Kutcher adolescent depression scale
with the Beck depression inventory. Journal of Child and Adolescent Psycho-
pharmacology, 12(2), 113e126. http://dx.doi.org/10.1089/104454602760219153.
Lemmens, J. S., Valkenburg, P. M., & Gentile, D. A. (2015). The internet gaming
disorder scale. Psychological Assessment, 72,567e582. http://dx.doi.org/10.1037/
pas0000 062.
Lemmens, J. S., Valkenburg, P., & Peter, J. (2009). Development and validation of a
game addiction scale for adolescents. Media Psychology, 12,77e95. http://
dx.doi.org/10.1080/15213260802669458.
Lemmens, J. S., Valkenburg, P. M., & Peter, J. (2011). The effects of pathological
gaming on aggressive behavior. Journal of Youth and Adolescence, 40,38e47.
Lubke, G., & Muth
en, B. (2004). Factor-analyzing Likert scale data under the
assumption of multivariate normality complicates a meaningful comparison of
observed groups or latent classes. Structural Equation Modeling, 11,514e534.
Marks, I. (1990). Behavioural (non-chemical) addictions. British Journal of Addiction,
85(11), 1389e1394.
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass
correlation coefficients. Psychological methods, 1(1), 30.
Meerkerk, G. J., Van den Eijnden, R. J. J. M., Vermulst, A. A., & Garretsen, H. F. (2009).
The compulsive Internet use scale (CIUS): some psychometric properties.
(continued )
…often used social media so you didn't have to think about unpleasant things?
…often used social media to escape from negative feelings?*
Problems
During the past year, have you …
…often not paid attention at school, while doing homework or at work because you were using social media?
…regularly not had enough sleep because you were using social media too late at night?
…regularly had arguments with others because of your social media use?*
Deception
During the past year, have you …
…regularly lied to your parents or friends about the amount of time you spend on social media?*
…regularly hidden your social media use from others?
…often used social media secretly?
Displacement
During the past year, have you …
…regularly devoted no attention to people around you (e.g. family or friends) because you were using social media?
…regularly had no interest in hobbies or other activities because you would rather use social media?
…regularly neglected other activities (e.g. hobbies, sport) because you wanted to use social media?*
Conflict
During the past year, have you …
…had serious problems at school or at work because you were spending too much time on social media?
…had serious conflict with your parent(s) and sibling(s) because of your social media use?*
…jeopardised or lost an important friendship or relationship because you were spending too much time on social media?
R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487486
CyberPsychology &Behavior, 12(1), 1e6.
Mehdizadeh, S. (2010). Self presentation 2.0: narcissism and self-esteem on Face-
book. Cyberpsychology, Behavior and Social Network Sites, 13,357e364.
Odaci, H., & Kalkan, M. (2010). Problematic Internet use, loneliness and dating
anxiety among young adult university students. Computers and Education, 55,
1091 e10 97.
Ophira, E., Nass, C., & Wagner, A. D. (2009). Cognitive control in media multitaskers.
In Proceedings of the national academy of sciences of the United States of America,
106. September 15, 2009. Available at: http://www.pnas.org/content/106/37/
15583.
Pantic, I. (2014). Online social networking and mental health. Cyberpsychology,
Behavior, and Social Networking, 17(10), 652e657.
Petry, N. M., Rehbein, F., Gentile, D. A., Lemmens, J. S., Rumpf, H., M€
oßle, T., et al.
(2014). An international consensus for assessing internet gaming disorder using
the new DSM-5 approach. Addiction, 109(9), 1399e1406.
Raju, N. S., Bilgic, R., Edwards, J. E., & Fleer, P. F. (1997). Methodology review: esti-
mation of population validity and cross-validity, and the use of equal weights in
prediction. Applied Psychological Measurement, 21(4), 291e305.
Rehbein, F., Kleimann, M., & M€
ossle, T. (2010). Prevalence and risk factors of video
game dependency in adolescence: results of a German nationwide survey.
Cyberpsychology, Behavior and Social Network Sites, 13, 269e277. http://
dx.doi.org/10.1089/cyber.2009.0227.
Rosenberg, M., Schooler, C., & Schoenbach, C. (1989). Self-esteem and adolescent
problems: modeling reciprocal effects. American sociological review,1004e1018.
Rumpf, H. J., Meyer, C., Kreuzer, A., John, U., & Meerkerk, G. J. (2011). Pr€
avalenz der
Internetabh€
angigkeit (PINTA). Bericht an das Bundesministerium für Gesundheit.
Verfügbar über.http://www.drogenbeauftragte.de/fileadmin/dateien-dba/
DrogenundSucht/Computerspiele_Internet/Downloads/PINTA-Bericht-
Endfassung_280611.pdf.
Russell, D., Peplau, L. A., & Cutrona, C. E. (1980). The revised UCLA Loneliness Scale:
concurrent and discriminant validity evidence. Journal of Personality and Social
Psychology, 39,472e480.
Ryan, T., Chester, A., Reece, J., & Xenos, S. (2014). The uses and abuses of Facebook: a
review of Facebook addiction. Journal of behavioral addictions, 3(3), 133e
148 .
Saaid, S. A., Al-Rashid, N. A. A., & Abdullah, Z. (2014). The impact of addiction to
Twitter among university students. In Future information technology (pp.
231e236). Springer Berlin Heidelberg.
Shotton, M. (1989). Computer addiction?: A study of computer dependency. London:
Taylor &Francis.
Soper, W. B., & Miller, M. J. (1983). Junk-time junkies: an emerging addiction among
students. The School Counselor, 31(1), 40e43.
Valkenburg, P. M., & Peter, J. (2011). Adolescents' online communication: an inte-
grated model of its attraction, opportunities, and risks. Journal of Adolescent
Health, 48, 121e127.
Van Rooij, A. J., Ferguson, C., Van de Mheen, D., & Schoenmakers, T. M. (2015).
Problematic Internet use: comparing video gaming and social media use
[Conference Abstract]. Journal of Behavioral Addictions, 4(1), 1e62. http://
dx.doi.org/10.1556/JBA.4.2015.Suppl.1.
Van Rooij, A., & Prause, N. (2014). A critical review of “Internet addiction”criteria
with suggestions for the future. Journal of behavioral addictions, 3(4), 203e213.
Van Rooij, A. V., & Schoenmakers, T. M. (2013). Monitor Internet en Jongeren 2010-
2012. Het (mobiele) gebruik van sociale media en games door jongeren [Monitor
Internet and Youth 2010-2012: The (mobile) use of social media and games by
adolescents]. http://www.ivo.nl/UserFiles/File/Publicaties/2013-01%20Factsheet
%20Social%20media%20en%20gamen.pdf.
Van Rooij, A. J., Schoenmakers, T. M., Van den Eijnden, R. J. J. M., & Van de Mheen, D.
(2010). Compulsive Internet use: the role of online gaming and other Internet
applications. The Journal of Adolescent Health, 47(1), 51e57.
Van Rooij, A. J., Schoenmakers, T. M., Van den Eijnden, R. J. J. M., Vermulst, A. A., &
Van de Mheen, D. (2012). Video game addiction test: validity and psychometric
characteristics. Cyberpsychology, Behavior, and Social Networking, 15(9),
507e511.
Van den Eijnden, R. J. J. M., Meerkerk, G., Vermulst, A. A., Spijkerman, R., &
Engels, R. C. M. E. (2007). Online communication, compulsive Internet use, and
psychosocial well-being among adolescents: a longitudinal study. Develop-
mental Psychology, 44, 655e665.
Van der Schuur, W. A., Baumgartner, S. E., Sumter, S. R., & Valkenburg, P. M. (2015).
The consequences of media multitasking for youth: a review. Computers in
Human Behavior, 53, 204e215.
Wallis, C. (2006). The multitasking generation. Time Magazine, 167(13), 48e55.
Wallis, C. (2010). The impacts of media multitasking on children's learning &devel-
opment. New York: Joan Ganz Cooney Center.
Wang, C., Lee, M. K. O., & Hua, Z. (2015). A theory of social media dependence:
evidence from microblog users. Decision Support Systems, 69,40e49.
Weinstein, A., & Lejoyeux, M. (2010). Internet addiction or excessive internet use.
The American journal of drug and alcohol abuse, 36(5), 277e283.
Wu, A. M. S., Cheung, V. I., Ku, L., & Hung, E. P. W. (2013). Psychological risk factors of
addiction to social networking sites among Chinese smartphone users. Journal
of Behavioral Addictions, 2(3), 160e166 .
Yen, J. Y., Ko, C. H., Yen, C. F., Wu, H. Y., & Yang, M. J. (2007). The comorbid psychiatric
symptoms of Internet addiction: attention deficit and hyperactivity disorder
(ADHD), depression, social phobia, and hostility. Journal of Adolescent Health,
41(1), 93e98.
Young, K. (1996). Internet addiction: the emergence of a new clinical disorder. In
Paper presented at the 104th annual meeting of the American Psychological As-
sociation. Toronto, Ontario, Canada.
Yu, C. Y., & Muth
en, B. (2002). Evaluation of model fit indices for latent variable models
with categorical and continuous outcomes (Technical Report). Los Angeles: Uni-
versity of California at Los Angeles, Graduate School of Education &Information
Studies.
R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487 487
