ArticlePDF Available

Abstract and Figures

HowNutsAreTheDutch (Dutch: HoeGekIsNL) is a national crowdsourcing study designed to investigate multiple continuous mental health dimensions in a sample from the general population (n = 12,503). Its main objective is to create an empirically based representation of mental strengths and vulnerabilities, accounting for (i) dimensionality and heterogeneity, (ii) interactivity between symptoms and strengths, and (iii) intra-individual variability. To do so, HowNutsAreTheDutch (HND) makes use of an internet platform that allows participants to (a) compare themselves to other participants via cross-sectional questionnaires and (b) to monitor themselves three times a day for 30 days with an intensive longitudinal diary study via their smartphone. These data enable for personalized feedback to participants, a study of profiles of mental strengths and weaknesses, and zooming into the fine-grained level of dynamic relationships between variables over time. Measuring both psychiatric symptomatology and mental strengths and resources enables for an investigation of their interactions, which may underlie the wide variety of observed mental states in the population. The present paper describes the applied methods and technology, and presents the sample characteristics.
Content may be subject to copyright.
HowNutsAreTheDutch (HoeGekIsNL): A
crowdsourcing study of mental
symptoms and strengths
LIAN VAN DER KRIEKE,
1
BERTUS F. JERONIMUS,
1
FRANK J. BLAAUW,
1,2
ROB B.K. WANDERS,
1
ANDO C. EMERENCIA,
1
HENDRIKA M. SCHENK,
1
STIJN DE VOS,
1
EVELIEN SNIPPE,
1
MARIEKE WICHERS,
1
JOHANNA T.W. WIGMAN,
1
ELISABETH H. BOS,
1
KLAAS J. WARDENAAR
1
&
PETER DE JONGE
1
1 University of Groningen, University Medical Center Groningen, Department of Psychiatry, Interdisciplinary
Center Psychopathology and Emotion regulation (ICPE), Groningen, The Netherlands
2 University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science, Distributed
Systems Group, Groningen, The Netherlands
Key words
mental symptoms, mental
strengths and resources,
longitudinal, ecological
momentary assessment,
crowdsourcing
Correspondence
Bertus F. Jeronimus, PO
Box 30.001, 9700 RB Groningen,
The Netherlands.
Telephone (+33) 050-3615655
Email: b.f.jeronimus@umcg.nl
Joint first authors.
Received 10 March 2015;
revised 10 July 2015;
accepted 17 August 2015
Abstract
HowNutsAreTheDutch (Dutch: HoeGekIsNL) is a national crowdsourcing
study designed to investigate multiple continuous mental health dimensions
in a sample from the general population (n= 12,503). Its main objective
is to create an empirically based representation of mental strengths and
vulnerabilities, accounting for (i) dimensionality and heterogeneity, (ii) interac-
tivity between symptoms and strengths, and (iii) intra-individual variability. To
do so, HowNutsAreTheDutch (HND) makes use of an internet platform that
allows participants to (a) compare themselves to other participants via
cross-sectional questionnaires and (b) to monitor themselves three times a
day for 30 days with an intensive longitudinal diary study via their smartphone.
These data enable for personalized feedback to participants, a study of proles of
mental strengths and weaknesses, and zooming into the ne-grained level of
dynamic relationships between variables over time. Measuring both psychiatric
symptomatology and mental strengths and resources enables for an investiga-
tion of their interactions, which may underlie the wide variety of observed
mental states in the population. The present paper describes the applied
methods and technology, and presents the sample characteristics. Copyright ©
2015 John Wiley & Sons, Ltd.
Introduction
The debate about the optimal way to dene and classify
mental health problems has intensied over the past three
decades (Kapur et al., 2012; Kendler and First, 2010;
Kendler et al., 2011; Wakeeld, 1992). The Diagnostic
and Statistical Manual of Mental Disorders (DSM) brought
standardization to a eld that used to be heavily
Copyright © 2015 John Wiley & Sons, Ltd. 123
International Journal of Methods in Psychiatric Research
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016)
Published online 22 September 2015 in Wiley Online Library
(wileyonlinelibrary.com) DOI: 10.1002/mpr.1495
fragmented, and allowed for a shared clinical language.
Nonetheless, DSM categories have also been criticized for
their lack of empirical support and the absence of an un-
derlying theoretical framework (Kapur et al., 2012;
Kendler et al., 2011; Wardenaar and de Jonge, 2013;
Whooley, 2014). Although the DSM system is mandatory
in psychiatric practice, scientists raised concerns about its
use, and argued that the current classication system ham-
pers our understanding of psychiatric disorders and can
lead to scientic stagnation (Dehue, 2014; Insel, 2013;
Kapur et al., 2012; Whooley, 2014). In the following we
propose three ways in which research can help to improve
the DSM system into an empirically based etiological
system.
First, the descriptive consensus-based DSM categories
imply a dichotomy of disordered versus healthy people,
namely subjects fulll a sufcient number of polythetic
diagnostic disorder classication criteria or they do not
(Kendler and Parnas, 2015; Krueger and Markon,
2006). Research suggests, however, that mental strengths
and symptoms are generally continuously distributed in
the population, without any evident zone of rarity,
and that existing cutoffs are arbitrary and inconsistent
(e.g. Gutiérrez et al., 2008; Kendell and Jablensky, 2003;
Kendler, 2012; Ormel et al., 2013; Widiger and Sankis,
2000). Mental health problems that might require care
can be located at the extreme ends of continuously distrib-
uted mental state dimensions (Clark and Watson, 1991;
Durbin and Hicks, 2014; Krueger, 1999; Mineka et al.,
1998). Although the dimensional approach to psychopa-
thology regains inuence in psychiatry (Dumont, 2010;
Kendler, 2012; Kendler and Parnas, 2015), research into
an empirical foundation remains imperative.
Second, the DSM denes mental health in terms of the
presence or absence of symptoms, which is a rather nar-
row (and negative) view on the numerous psychological
processes that dene a persons mental state (Duckworth
et al., 2005; Horwitz and Wakeeld, 2007; Huber et al.,
2011; Kendler, 2012; Sheldon et al., 2011; Whooley,
2014). Focusing solely on mental illness can, at best, re-
duce mental illness, but does not sufce for mental health
(Keyes, 2007; Westerhof and Keyes, 2010). Mental health
requires the absence of disease and disability (negative
states), social and psychological well-being (positive
states), and abilities to adapt to ones environment and
self-manage (World Health Organization, 1946; Huber
et al., 2011; Keyes, 2007; Solomon, 2014). More funda-
mentally, it is unlikely that we learn to understand the
mechanisms underlying psychiatric symptomatology
when we fail to take into account the role of mental
strengths, resources, and contextual factors in the eventual
(non)expression of mental health problems, such as
humour, self-acceptance, hope, social participation, and
social support (Duckworth et al., 2005; Keyes, 2007;
Larson, 1999; Luthar et al., 2000; Seery et al., 2010;
Seligman and Csikszentmihalyi, 2000; Sheldon et al.,
2011; Westerhof and Keyes, 2010).
Third, while some DSM criteria already specify vari-
ability between and within individuals, such as having
symptoms most of the day, nearly every dayversus
most days, these specications lack a solid empirical
foundation, and do not allow for the identication of
course uctuations (Horwitz and Wakeeld, 2007;
Hyman, 2007; Kapur et al., 2012; Kupfer et al., 2002;
Wardenaar and de Jonge, 2013; Widiger and Samuel,
2005), or for sequential expressions, such as a shift from
sadness to anxiety over time (Doré et al., 2015; Kessler
et al., 2005; Stossel, 2013). While DSM categories are pre-
sented as homogenous disease entities, combinations of
symptoms prevail (co-morbidity), while the boundaries
between diagnostic categories are necessarily fuzzy (Clark
et al., 1995; Kendler, 2012, Krueger and Markon, 2006;
Ormel et al., 2013; Widiger and Samuel, 2005; 2008). Ad-
ditionally, treatment effects tend to be rather non-specic
(Roest et al., 2015), and even genetic predispositions defy
DSM syndrome boundaries in twin (Kendler, 1996), fam-
ily (Dean et al., 2010), and genome-wide association stud-
ies (Psychiatric Genomics Consortium, 2015).
Taken together, as long as the DSM is unable to take
the dynamic interactions between mental symptoms,
strengths, and contextual factors into account, and lacks
an empirically based identication of between and within
individual variation, our research on the classication of
mental disorders remains unrealistic. This may help ex-
plain why research into disease mechanisms is stagnant
and difcult to replicate.
The current project
The research project HowNutsAreTheDutch (Dutch:
HoeGekIsNL) is designed to allow for the investigation
of mental health as a dimensional and dynamic phe-
nomenon, characterized by both vulnerabilities and
strengths. HowNutsAreTheDutch (henceforth HND) is
a widely broadcasted national crowdsourcing study in the
Netherlands collecting self-report data on mental health
in a general population sample. The project uses an inter-
net platform to recruit participants and invite them to as-
sess themselves on multiple mental health dimensions.
The combination of a cross-sectional and intensive longi-
tudinal diary design allows for a data-driven empirical ap-
proach that may help us generate new ideas about how key
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.124
variables interact in a multidimensional representation of
individualsmental condition. The primary purpose of
HND is to explore the associations and dynamic interac-
tions between mental strengths and vulnerabilities, both
between and within participants. This paper presents the
objectives, methods, and technology of both the cross-
sectional and longitudinal part of the HND study, and pro-
vides an overview of the samples characteristics in terms of
demographics, psychometrics, and missing data.
Methods
Crowdsourcing procedure
For HND we applied a crowdsourcing procedure, an im-
portant model for doing psychological research in which
a task is outsourced to a group of people, often online,
in an open call (Brabham, 2008; Howe, 2006). Previous
health research studies have used crowdsourcing as a
method to collect new information, and showed that it
can be a useful method to obtain information that other-
wise tends to be overlooked by researchers (e.g.
Bevelander et al., 2014), or to collect big datasets on mul-
tiple outcomes that could not be realized without the par-
ticipation of a large crowd (e.g. Revelle et al., 2010).
The crowdsourcing method enables for the develop-
ment of citizen science, in which the general public vol-
unteers to assist scientists in their research activities and
contribute with their intellectual effort, knowledge, or
tools and resources to answer real-world questions (Hand,
2010). We hope that the HND crowdsourcing approach
(a) engages Dutch inhabitants with the debate about psy-
chiatric classication (Dehue, 2014) and (b) results in a
sizeable sample of participants that allows for data-driven
analyses of the relations between mental symptoms and
strengths both within and between individuals.
With HND we launched an open call to inhabitants of
the Netherlands to join our research, and invited them to
visit the Dutch website www.HoeGekIs.nl (also www.
HowNutsAreTheDutch.com), which has been online since
19 December 2013. The open call was announced on local
and national radio broadcasts, television, during local po-
dium discussions, in newspapers, and in magazines. The
news about the HND research project was picked up and
further disseminated via online blogs, twitter, and other
social media.
To join the project, participants had to register and cre-
ate an account. Participants lled out their email address
and a password on www.HoeGekIs.nl and received an
email with a hyperlink to conrm their account. Before
starting the actual research, participants were asked to
provide information about their gender, birth year and
month, their postal code area, and country of residence
(the Netherlands/Belgium/Other). Although HND is
targeted on Dutch citizens, we added a question about
country of residence after news about HND was picked
up by the Belgium media and Dutch speaking participants
from Belgium started to join the website.
Two studies
The HND website comprises a cross-sectional study and a
longitudinal study with intensive repeated assessments in
daily life, namely a diary study with ecological momentary
assessments (EMA, see Bolger et al., 2003). Participants
could complete either one of these studies, or both.
Cross-sectional study
Procedure and questionnaires. The cross-sectional study
was launched together with the website on 19 December
2013. In this study participants were invited to complete
various questionnaire modules, that is, a questionnaire
or a set of combined questionnaires covering a specic
domain (see Table 1). The order in which the modules
could be completed was partly xed. All questionnaire
modules were visible from the start, but initially only part
of them was activated. The rst mandatory module was
the Startmodule, assessing participantssocio-
demographic prole. Subsequently, participants got access
to three key modules; (i) (an extensive assessment of
ones) Living situation; (ii) Affect/mood; (iii) Well-being,
which could be completed in any order. After the
Affect/mood and Well-being modules had been com-
pleted all other modules became available and could be
completed in any order. These latter modules were not
yet available at the launch of the HND website, but were
added to the website one at a time over the year following
the launch. Every three months participants were in-
formed about these additions via an email newsletter.
All implemented are outlined in Table 1. Eligible partici-
pants were aged 18 or older and consented to their data
being used for research.
Feedback generation. After completing a questionnaire
module participants received instant and automated feed-
back on the website. This feedback consisted of bars show-
ing their scores relative to the maximum possible score on
a given questionnaire and bars or spider plots reecting
their personal scores relative to the average scores of the
HND participants. Examples are presented in Figures
S1S3 in the Supplementary Material.
Krieke et al. HowNutsAreTheDutch Crowdsourcing Study
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd. 125
Table 1. Modules, instruments, and contents of the cross-sectional study
Module Instrument Description Items
Response
range Authors
Start Gender, birth year, postal code area, relationship status, number
of children, education level, and occupational situation.
8
Living situation
(socio-demography)
Country of origin (also for both parents), family income, living
arrangement, family members, pets, religion, time spent on
television/internet/sports, length and weight, and hand preference.
17
Affect/mood PANAS PANAS Flemish version, assesses 10 positive and 10 negative
emotions over the past week.
20 15 Peeters et al., 1996;
Raes et al., 2009
QIDS The QIDS assesses and classies DSM major depression with
nine domains, sad mood, concentration, self-criticism, suicidal
ideation, interest, energy/fatigue, sleep, change in appetite/
weight, psychomotor, over the past week.
16 03 Rush et al., 2003, 2006
DASS The DASS measures mood over the past week and is sensitive to
subthreshold symptoms.
42 03 Lovibond and Lovibond,
1995a, 1995b
De Beurs et al., 2001
Well-being MANSA The MANSA measures quality of life on multiple domains, with 12
items for a sum score, and four additional yes/no items.
16 17 Priebe et al., 1999
Priebe et al., 2010
Happiness
index
The Happiness index assesses the degree to which one judges
the quality of ones life in a single item: Do you feel happy in
general?
1010 Fordyce, 1988;
Veenhoven, 1994;
Abdel-Khalek, 2006
SPF-IL The SPF-IL measures the ve universal primary goals affection,
behavioural conrmation, status, comfort, and stimulation, which,
according to SPF theory, underlay individual well-being.
15 03 Nieboer et al., 2005
Ryff scales The Ryff scales of psychological well-being measure self-
acceptance, positive relations with others, autonomy,
environmental mastery, purpose in life, and personal growth.
39 16 Van Dierendonck,
2004
Personality NEO-FFI-3 The NEO-FFI-3 personality inventory assesses the Big Five
personality domains Neuroticism, Extraversion, Openness to
experience, Agreeableness, and Conscientiousness with 60
items. This instrument was extended with 36 neuroticism items
from the NEO-PI-3 to derive all facet traits for the neuroticism
domain.
96 15 De Fruyt and Hoekstra,
2014
Dark Triad The Dark Triad assesses tendencies towards Narcissism,
Machiavellianism, and Psychopathy.
12 19 Paulhus and Williams, 2002,
Klimstra et al., 2014
(Continues)
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.126
Table 1. (Continued)
Module Instrument Description Items
Response
range Authors
Doing, Feeling,
Thinking
Doing, Feeling, Thinking assesses tendencies towards the
behavioural styles Doing (practically-oriented), Feeling (relation-
oriented), and Thinking (content-logic-oriented).
913 De Klerk et al., 2003
Somatic Symptoms PHQ-15 The PHQ-15 is a screening and diagnostic tool to assess somatic
symptoms associated with mental health disorders.
15 13 Kroenke et al., 2002
RoSi A composite of ve items from the SCL somatic scale and eight
items derived from an expert committee supervised by J.
Rosmalen.
13 02 Arrindell and Ettema,
1986 (SCL)
Whiteley Index The Whiteley index measures tendencies towards hypochondria. 14 01 Speckens et al., 1996
Psychotic Experiences CAPE The CAPE measures positive and negative symptoms of
psychosis with 42 items. We only selected the 34 items about
psychotic symptoms and skipped the depression items.
34 03 Konings et al., 2006
Humour HSQ The HSQ assesses afliative, self-enhancing, aggressive, and
self-defeating humour styles.
32 17 Martin et al., 2003
Optimism LOT-R The LOT-R assesses dispositional optimism (and pessimism). 10 04 Glaesmer et al., 2012
Empathy EQ The EQ questionnaire measures both affective empathy via
shared emotions and cognitive empathy or theory of mind.
40 02 Baron-Cohen and
Wheelwright, 2004
Childhood adversity CTQ-SF The CTQ-SF is a retrospective self-report questionnaire designed
to assess ve dimensions of childhood maltreatment: physical
abuse, emotional abuse, sexual abuse, physical neglect, and
emotional neglect.
28 15 Thombs et al., 2009
Intelligence ICAR We selected 11 items measuring inductive reasoning and 24 items
measuring three-dimensional (3D) rotation abilities from the ICAR
cognitive item pool.
35 1618 Condon and Revelle,
2014
Evaluation Evaluation The evaluation questionnaire was designed to evaluate the HND
website, the cross-sectional questionnaire modules, the feedback
upon completed modules, and the impact of participation.
7110
Note: CAPE = Community Assessment of Psychic Experiences; CTQ-SF = Childhood Trauma Questionnaire Short Form; DASS = Depression Anxiety Stress Scale;
DSM = Diagnostic and Statistical Manual of Mental Disorders; EQ = Empathy Quotient; HND = HowNutsAreTheDutch; HSQ = Humour Styles Questionnaire;
ICAR = International Cognitive Ability Resource Base; LOT-R = Life Orientation Test Revised; MANSA = The Manchester Short Assessment of quality of life;
NEO-FFI-3 = The NeuroticismExtraversionOpenness Five-Factor Inventory updated and revised version; PANAS = Positive And Negative Affect Schedule; PHQ-
15 = The Patient Health Questionnaire 15 item version; SCL = Symptom Checklist; SPF = Social Production Functions; SPF-IL = The SPF Instrument for the Level of
well-being; QIDS = The Quick Inventory of Depressive Symptomatology; RoSi = Rosmalen Somatic items scale.
Krieke et al. HowNutsAreTheDutch Crowdsourcing Study
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd. 127
Diary study
Procedure. The diary study was launched on 22 May 2014. In
this study participants were intensively monitored in their
natural environments by means of electronic diaries, three
times a day for 30 days, resulting in a maximum of 90 assess-
ments per individual. Assessments were prompted at equi-
distant time points with a six-hour interval in between,
with the exact time points depending on participants
sleepwake schedule. Participants received a text message
on their mobile phone with a link to a questionnaire. They
were asked to ll out the questionnaire immediately after
the alert, or, if impossible, within one hour, after which
the questionnaire could no longer be accessed. Participants
were informed about the requirements and procedure of
the diary study by means of an information page and a short
animated movie clip on the HND website. Additionally, they
could download an information booklet with details on the
study procedure, diary items, and their reward, being a dig-
ital report containing personalized feedback.
The study requirements were: age 18 or above, having a
mobile phone with internet connection, not engaged in
shift work, not anticipating a major disruption of daily
routines (e.g. a planned trip abroad, an anticipated surgi-
cal operation), being aware that participation would be
useless in case too many assessments would be missed,
and approving of ones anonymous data being used for
scientic research. Participants had to check a box for each
of these requirements before they could proceed. Subse-
quently, they had to complete a baseline assessment
consisting of the items of the positive and negative affect
schedule (PANAS; Peeters et al., 1996; Raes et al., 2009),
the Quick Inventory of Depressive Symptoms (QIDS;
Rush et al., 2003, 2006), and two extra items retrieved
from the Inventory of Depressive Symptomatology (IDS;
Rush et al., 1996) to assess anxiety/panic symptoms. Fi-
nally, participants congured their personal settings for
the daily assessments, i.e. their telephone number, their
preferred start date (within ve days after completion of
the baseline assessment), and the sampling schedule. Par-
ticipants were instructed to pick a sampling schedule that
tted their daily rhythm, with the evening measurement
preferably half an hour before their regular bedtime.
After completion of their diary study, participants were
sent a short evaluation questionnaire. Participants who
quit the study prematurely were asked for their reasons
to quit by means of a short questionnaire.
Questionnaire items. The diary questionnaire contained 43
items. It combined items from existing and validated ques-
tionnaires and a few newly created items. We assessed
subjective well-being, sleep, mood, anxiety, depression,
physical activity, physical discomfort, self-esteem, worry-
ing, loneliness, mindfulness, context (location, social com-
pany, activities), and the appraisal of this context, stressful
events, time pressure, the feeling one makes a difference,
laughing, and being outdoors. All questionnaire items
and literature references are presented in Table 2. Addi-
tionally, participants could dene a personal item that they
felt relevant to their situation. This item could be chosen
from a list of options or could be self-created during the
conguration of personal settings. Examples of personal
items were: I worry a lotor I smoked a lot since the last
assessment. All items except categorical ones were rated
on a visual analogue scale (VAS) ranging from 0 to 100,
with appropriate labels at the extremes and middle of the
scale, and the middle as default positive. To answer a ques-
tion the slider had to be moved.
Feedback generation. After completion of the study partic-
ipants received instant and automated feedback on the
website. Participants who completed at least 65% (t59)
of the assessments received basic personalized feedback
consisting of graphs and explanatory text (see Figures
S4S7 in the Supplementary Material). Participants who
completed at least 75% (t68) of the assessments also re-
ceived personal network models showing the interrela-
tionships between variables assessed in the diary study,
see Figure 1 for an example network. The rst network
model depicts the concurrent relationships between vari-
ables. This model shows how a participants affect, cogni-
tions, and behaviours are related to each other at the same
moment in time. The second network model shows the
dynamic, directed relationships between variables, indicat-
ing how they affected each other over time.
The network models were estimated for each partici-
pant separately using vector autoregressive (VAR) model-
ling (Brandt and Williams, 2007; Lütkepohl, 2006). To
automate this procedure, we used Autovar, an open source
R package that reads raw data and automatically ts and
evaluates VAR models (Blaauw et al., 2014; Emerencia
et al., 2015, Van der Krieke et al., 2015. The number of
variables included in the network models was limited to
six, for the sake of comprehension and in order to focus
on the most informative results. These six variables were
selected based on highest moment-to-moment variability
(as indicated by the mean squared successive difference,
MSSD) and lowest skewness. Both variables usually per-
ceived as positive(e.g. laughing, relaxation) and vari-
ables perceived as negative(e.g. rumination, feeling
down) were selected, as well as the participants personal
item, unless variability was too low (MSSD <50) or
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.128
Table 2. Items of the diary study
Question Dutch Translation Response range Range Description
1. Hoe gaat het op dit
moment met u?
How are you doing right
now?
Very badto Very good0100 Moment-to-moment quality of
life
2. Heeft u sinds het vorige
meetmoment geslapen?
Did you sleep since the
last measurement?
(1) No 12 Sleep (check boxes). If yes,
go to 34.(2) Yes
3. Heeft u goed geslapen? Did you sleep well? Not at allto Very well0100 Quality of sleep
4. Heeft u lang genoeg
geslapen?
Did you sleep long
enough?
Too shortto Too long0100 Duration of sleep
5. Ik voel me ontspannen I feel relaxed Not at allto Very much0100 Positive affect Deactivation
6. Ik voel me somber I feel gloomy Not at allto Very much0100 Negative affect Deactivation
7. Ik voel me energiek I feel energetic Not at allto Very much0100 Positive affect Activation
8. Ik voel me angstig I feel anxious Not at allto Very much0100 Negative affect Activation
9. Ik voel me enthousiast I feel enthusiastic Not at allto Very much0100 Positive affect Activation
10. Ik voel me onrustig I feel nervous Not at allto Very much0100 Negative affect Activation
11. Ik voel me tevreden I feel content Not at allto Very much0100 Positive affect Deactivation
12. Ik voel me prikkelbaar I feel irritable Not at allto Very much0100 Negative affect Activation
13. Ik voel me kalm I feel calm Not at allto Very much0100 Positive affect Deactivation
14. Ik voel me lusteloos I feel dull Not at allto Very much0100 Negative affect Deactivation
15. Ik voel me opgewekt I feel cheerful Not at allto Very much0100 Positive affect Activation
16. Ik voel me moe I feel tired Not at allto Very much0100 Negative affect Deactivation
17. Ik ervaar lichamelijk
ongemak
I experience physical
discomfort
Not at allto Very much0100 Somatic symptoms
18. Ik voel me gewaardeerd I feel valued Not at allto Very much0100 Self-esteem
19. Ik voel me eenzaam I feel lonely Not at allto Very much0100 Loneliness
20. Ik heb het gevoel tekort
te schieten
I feel I fall short Not at allto Very much0100 Worthlessness
21. Ik voel me zelfverzekerd I feel condent Not at allto Very much0100 Self-esteem
22. Ik pieker veel I worry a lot Not at allto Very much0100 Worrying
23. Ik ben snel afgeleid I am easily distracted Not at allto Very much0100 Concentration/mindfulness
24. Ik vind mijn leven de
moeite waard
I feel my life is worth
living
Not at allto Very much0100 Worthlessness/suicidal
ideation
25. Ik ben van slag I am unbalanced Not at allto Very much0100 Stress reactivity
26. Ik leef in het hier en nu I am in the here and now Not at allto Very much0100 Mindfulness
27. Mijn eetlust is My appetite is Much smaller than usualto
Much larger than usual
0100 Appetite
(Continues)
Krieke et al. HowNutsAreTheDutch Crowdsourcing Study
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd. 129
Table 2. (Continued)
Question Dutch Translation Response range Range Description
28. Hoe druk heb ik het? How busy am I? (1) Much too busy 15 Time pressure
(2) Pleasantly busy
(3) Neutral
(4) Pleasantly quiet
(5) Much too quiet
29. Waar was ik het
afgelopen dagdeel de
meeste tijd?
Where have I spent
most of my time since
the last measurement?
(1) At home 19 Location (check boxes; only
one location could be
checked)
(2) At work/school
(3) With family/friends
(4) On the way
(5) Vacation home/hotel/
camping
(6) Hospital/health facility
(7) Restaurant/beanery
(8) In nature
(9) Somewhere else
30. Wat deed ik het
afgelopen dagdeel de
meeste tijd?
How did I spend most of
my time since the last
measurement?
(1) Resting/sleeping 113 Activities (check boxes; only
one activity could be
checked)
(2) Household/groceries
(3) Working/studying/
volunteering
(4) Exercising/walking/cycling
(5) Yoga/meditation/ sauna
visit etc.
(6) Reading
(7) Hobby (e.g. gardening,
making music)
(8) Trip (e.g. leisure park,
concert)
(9) Watching tv
(10) Websurng/ gaming/ social
media
(11) Conversing
(12) Something intimate (e.g.
Cuddling, sex)
(13) Something else/ all kinds of
things
(Continues)
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.130
Table 2. (Continued)
Question Dutch Translation Response range Range Description
31. Ik ervoer deze activiteit
overwegend als
I experienced this
activity mainly as
Very unpleasant, via Neutral,
to Very pleasant
0100 Appraisal of activity
32. Is er het afgelopen
dagdeel iets bijzonders
gebeurd?
Did something special
happen since the last
measurement?
(1) No, nothing 14 Special event (check boxes;
only one box could be
checked). If nothing, jump to
34, otherwise 33.
(2) Yes, something positive
(3) Yes, something neutral
(4) Yes, something negative
33. Waar had dit mee te
maken?
This was related to (1) Myself 17 Context of special event
(check boxes; only one box
could be checked).
(2) Home situation/ close family/
signicant others
(3) Friends/ other family/
acquaintances
(4) Work/ school
(5) Society/ news
(6) Public space/ strangers
(7) Other
34. Ik was het afgelopen
dagdeel grotendeels
Most of the time since
the last measurement I
was
(1) Alone 12 Social company (check
boxes; only one box could
be checked). If alone, go to
35, followed by 38. If in
company, go to 3637.
(2) In company
35. Ik was liever in
gezelschap geweest
I would rather have
been with others
No, preferably notto Yes,
certainly
0100 Appraisal of being alone
36. Ik zou liever alleen zijn
geweest
I would rather have
been alone
No, preferably notto Yes,
certainly
0100 Appraisal of social company
37. Ik vond dit gezelschap
overwegend
I found my company
predominantly
Very unpleasant, via Neutral,
to Very pleasant
0100 Appraisal of social company
38. Ik heb in het afgelopen
dagdeel gelachen
Since the last
measurement I had a
laugh
Not at allto Very much0100 Laughing
39. Ik heb in het afgelopen
dagdeel iets voor
iemand kunnen
betekenen
Since the last
measurement I was
able to make a
difference
Not at allto Very much0100 Making a difference
(Continues)
Krieke et al. HowNutsAreTheDutch Crowdsourcing Study
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd. 131
Table 2. (Continued)
Question Dutch Translation Response range Range Description
40. Ik ben het afgelopen
dagdeel buiten geweest
Since the last
measurement I have
been outdoors
Not at allto Very much0100 Being outside
41. Hoe lichamelijk actief
was ik het afgelopen
dagdeel?
Since the last
measurement I was
physically active
Not at allto Very much0100 Physical activity
42. Ik deed dingen op de
automatische piloot,
zonder me bewust te
zijn van wat ik deed
I did jobs or tasks
automatically without
being aware of what I
was doing
Not at allto Very much0100 Mindfulness
43. Mijn eigen belangrijke
factor
My personal important
factor
Not at allto Very much0100 Personal item
Note: Momentary affect and anxiety were assessed with 12 items from the circumplex model of affect, which describes affect in terms of two dimensions: valence and
activation (items 516; Barrett and Russell, 1998; Yik et al., 1999). Sleep was assessed with one item from the Pittsburgh Sleep Diary (Monk et al., 1994) and two self-
constructed items (items 24). We added four extra items reecting the DSM-V criteria for depression not already covered by the affect and sleep items, which we
derived from the Patient Health Questionnaire-9 (Wittkampf et al., 2009) and the Inventory of Depressive Symptomatology (Rush et al., 1996), and adapted for daily
use (items 20, 23, 24, 27). Subjective well-being was assessed with a single item reecting moment-to-moment quality of life (item 1: Barge-Schaapveld et al., 1999).
Self-esteem, worrying, loneliness, physical activity, physical discomfort, context (location, social company, activities), and the appraisal of this context were assessed
with items adapted from previous ecological momentary assessment studies (items 1719, 21, 22, 2931, 3437, 42: Barge-Schaapveld et al., 1999; Collip et al.,
2011; Csikszentmihalyi and Larson, 1987; Doane and Adam, 2010; Myin-Germeys et al., 2009; Savin-Williams and Demo, 1983; Wichers et al., 2011). Mindfulness
was assessed with two items adapted from the Five Facet Mindfulness Questionnaire (Baer et al., 2006, 2008) and one self-constructed item (items 23, 26, 42).
We added seven self-constructed items to assess stressful events, time pressure, making a difference, laughing, and being outside (items 25, 28, 32, 33, 38, 39,
40). Participants could also dene a personal item that they felt relevant to their situation (item 43; see Methods section).
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.132
skewness too high (z-skewness >4). Only VAR models
with one time lag were estimated, to prevent over-
parameterization. Dummy variables for the part of the
day (morning/afternoon/evening) were included in all
models. Trend variables denoting the assessment point
(and the square of it) were included if necessary. Dummy
variables for the days of the week were included and vari-
ables were log transformed if this improved model t.
Autovar only considered models that met the assumptions
of stability, normality, homoscedasticity, and indepen-
dence, and selected the best model based on the Akaike
Information Criterion. Details on the procedure for auto-
mated network models are available from the authors upon
request.
Technological infrastructure of HowNutsAreTheDutch
(HND)
An architectural overview of the HND web application,
the external services involved, and the security of the con-
nections is presented in Figure 2. The gure shows that
HND consists of three main components. The rst
Figure 1. Example of a personal network model showing concurrent (left) and dynamic relationships (right) between diary
items. Note: Red nodes represent variables that tend to be perceived as negative (e.g. loneliness, sadness). Green nodes
represent variables that tend to be perceived as positive (e.g. relaxation, mindfulness, feeling cheerful). The blue node rep-
resents the personal variable that participants could choose to add to the diary assessment. This variable could either be
negativeor positiveand could be different for each participant. The size of the node indicates its relative importance
(i.e. the bigger a node, the more connections the variable has with other variables). The lines represent the connections
between variables; the thickness indicates the strength of the relationship. A plus refers to a positive relationship; a minus
refers to a negative relationship. The arrowheads (only in the dynamic networks) indicate the direction of the relationships.
Figure 2. Architectural overview of the HowNutsAreTheDutch (HND) web application. Note: The cylindrical shapes repre-
sent databases. The rectangular shapes depict (web) services. The hatched rectangle is the actual HND web application that
serves the website.
Krieke et al. HowNutsAreTheDutch Crowdsourcing Study
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd. 133
component is the HND application itself. This application
is implemented in the Ruby on Rails framework (http://
rubyonrails.com). The data is stored in two databases.
The personal socio-demographic data (i.e. gender, birth
month, birth year, and postal code area) are stored in a
PostgreSQL database (http://postgresql.org). The other
questionnaire data are stored in a MongoDB database
(http://mongodb.com). The use of two separate databases
lowers the impact of a potential database security breach.
Both databases would have to be compromised to trace
questionnaire data back to potentially identifying personal
information. All web trafc to and from HND is encrypted
using a 128bit TLS/2048bit RSA Secure Socket Layer (SSL)
certicate. This SSL certicate ensures that any data ex-
changed between participants and the HND website appli-
cation is unreadable for anyone else. The HND component
in Figure 2 serves the web application to the clients, con-
tains the registration and login services, provides the
cross-sectional questionnaires and provides the feedback.
This component also interfaces to the other two compo-
nents; the RoQua service and the Autovar service.
RoQua is a Software as a Service (SAAS) product that of-
fers scheduling of momentary assessments and collection of
questionnaire data (http://roqua.nl). HND communicates
with RoQua in order to schedule the measurement for each
participant. When a participant is scheduled to ll out a
questionnaire, RoQua noties HND. HND then noties
the participants and redirects them to the RoQua service,
which conducts the questionnaire and stores the result.
The Autovar service is a statistical service used to ana-
lyse the diary questionnaire data (http://autovar.nl). In or-
der to expose Autovars functions to the HND platform,
Autovar uses a service known as OpenCPU (Ooms,
2014). OpenCPU offers a RESTful (REpresentational State
Transfer) interface that allows R functions to be used by
other systems and programming languages.
Ethics
The HND study protocol was assessed by the Medical
Ethical Committee of the University Medical Centre Gro-
ningen. The committee judged the protocol to be exempted
from review by the Medical Research Involving Human
Subjects Act (in Dutch: WMO) because it concerned a
non-randomized open study targeted at anonymous volun-
teers in the general public (registration number M13.147422
and M14.160855).
Sample comparisons
One has to realize that particular individuals are less likely
to participate in crowdsourcing studies. To explore
selection effects we compared the characteristics of the
HND sample with (a) the general Dutch population
according to the Dutch Governmental Agency for Statis-
tics (CBS) and (b) two representative samples from the
non-institutionalized Dutch population, that is, the
Netherlands Mental Health Survey (NEMESIS-2,
n= 6646, 20072009, see de Graaf et al., 2010) and the
Lifelines population study from the north of the Nether-
lands (n= 167,729, 20062013, see Scholtens et al., 2014).
Results
Cross-sectional study
Sample characteristics
Up to 13 December 2014 12,734 participants participated
in the cross-sectional study. We excluded 231 participants
from our analyses because they were younger than 18
(n= 228) or provided unrealistic entries (e.g. birth
year <1900; n= 3), resulting in a nal sample of 12,503.
The mean age of the participants was 45 years [standard
deviation (SD) = 15] and 65% were women. A detailed
overview of the participants is given in Table 3. Participants
were sampled from all regions of the Netherlands, as illus-
trated by the heat map of the Netherlands in Figure 3.
The coverage concurs very well with population density
scores. However, compared to the Dutch population, the
HND participants were more often women (65.2% versus
50.5% in the population, NEMESIS = 55.2%, Life-
lines = 57.9%), on average slightly older (45 versus
39 years; NEMESIS = 44, Lifelines = 42), more often with
a romantic partner (74% versus 58%), with whom they
cohabited more often (61% versus 47%; NEME-
SIS = 68%). Most saliently, HND sampled few people from
lower educated strata (2% versus 22%; NEMESIS = 5%),
as well as medium education levels (16 years and more,
22% versus 43%; NEMESIS = 60%); HND participants
tend to be higher educated (>20 years, 76% versus 35%;
NEMESIS = 35%). Elderly were relatively well sampled in
HND, as most participants were older than 45 (55%),
and 9% of them were older than 65 (versus 19% of the
population; Lifelines = 7.6%, NEMESIS did not include
participants older than 64). To enable comparisons be-
tween this HND sample and population samples, and to
perform robustness checks for our models, we calculated
a selection bias weight factor, based on population propor-
tions derived from the CBS. Post-stratication weights
were derived for 36 strata based on age (six categories),
gender, and education level (three categories), see Supple-
mentary Material Table S1. Our weighted results are pre-
sented in Table 3.
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.134
Key results
Exactly 62,068 questionnaire modules were completed, on
average ve modules per participant (12,402 participants
lled out 1 of the 14 available questionnaires). The key
questionnaire modules focusing on Affect/mood and
Well-being were completed approximately 8000 and
10,000 times, respectively (see Supplementary Material
Table S2), while 5144 participants lled out all three key
modules (including life situation). All modules were com-
pleted by 627 participants (except intelligence, which was
only available in the last three weeks).
As presented in Table 3, on average participants reported
more positive than negative affect (PANAS, mean = 34 ver-
sus 20, respectively, both SD = 7 and range 1050). How-
ever, both the positive affect and the negative affect scales
showed high variance. A visual representation of the rela-
tionship between positive and negative affect in Figure 4 in-
dicates that participants with a similar score on negative
affect varied substantially in their positive affect scores, and
the other way around, despite their strong correlation
(r=0.52, p<0.001). Albeit women reported slightly more
negative (t=8.25,p<0.001, d= 0.19) and less positive affect
than men (t=4.54, p<0.001, d= 0.11) gender differences
were minimal (see Supplementary Material Table S3).
Regarding mood, the DepressionAnxietyStress Scale
(DASS) showed that symptoms of psychological stress were
most common, followed by symptoms of depression and
anxiety. The large standard deviations and broad ranges of
these scales indicate that substantial heterogeneity exists
among participants. Based on DASS cutoff values (Lovibond
and Lovibond, 1995b), 14.8% of the participants in our
sample reported mild, 9.1% moderate, 3.9% severe, and
1.3% extremely severe depression symptom levels (in the
past week). With respect to anxiety, 15.1% scored mild,
11.0% moderate, 4.6% severe, and 1.9% extremely severe.
With respect to psychological stress, 17.9% scored mild,
9.5% moderate, 2.7% severe, and 0.4% extremely severe.
Table 3. Baseline characteristics of the cross-sectional HowNutsAreTheDutch (HND) sample
Module Topic N Range
Raw descriptives Weighted descriptives
a
SDMean SE SD N
b
Mean SE
c
Start Age 12,503 18 to 90
d
45.3 0.13 14.6 12,189 40.7 0.17 13.7
Education level 12,189 1 to 8
e
6.9 0.01 1.2 12,189 6.4 0.02 1.5
Duration of romantic
relationship in years
9,038 1 to 80 18.3 0.14 13.7 9,038 14.7 0.18 12.6
Living situation
(socio-demography)
Number of children 12,190 0 to 12 1.2 0.01 1.2 12,189 1.1 0.01 1.2
Height in centimetres
f
11,035 100 to 213 174.7 0.09 9.1 11,034 175.1 0.12 9.1
Weight in kilograms
f
11,034 20 to 190 74.6 0.14 14.7 11,034 74.7 0.20 15.2
Affect/ Mood PANAS Positive affect 8,031 10 to 50 34.2 0.08 6.9 8,030 33.5 0.11 7.1
PANAS Negative affect 8,032 10 to 50 19.7 0.08 7.2 8,030 20.5 0.12 7.4
DASS Depression 7,972 0 to 42 6.8 0.09 7.8 7,972 7.3 0.13 8.2
DASS Anxiety 7,972 0 to 42 3.6 0.06 4.9 7,972 4.0 0.09 5.4
DASS Distress 7,973 0 to 42 8.6 0.08 7.0 7,972 9.4 0.12 7.3
Well-being MANSA quality of life 10,181 12 to 84 62.1 0.09 8.6 10,180 61.4 0.13 9.0
Happiness index 10,152 0 to 10 6.9 0.02 1.6 10,151 6.8 0.02 1.7
SPF-IL 10,131 0 to 45 25.1 0.06 5.9 10,130 24.4 0.08 5.9
Ryff total 10,033 46 to 234 166.6 0.27 26.6 10,133 163.7 0.38 27.4
Note: DASS = Depression-Anxiety-Stress Scale; MANSA = The Manchester Short Assessment of quality of life;
PANAS =Positive and Negative Affect Schedule; SPF-IL =The Social Production Function Instrument for the Level of well-being.
a
The calculation of the post-stratication weights can be found in Supplementary Material Table S2.
b
The Nfor the weighted descriptives (N= 12,189) is smaller than for the raw descriptives because 314 participants (2.5%) did
not provide their education level correctly.
c
Standard errors based on Taylor series linearization in R-package svy(Lumley, 2014).
d
The distribution of age and gender is presented visually in Figure S8 in the Supplementary Material.
e
Education level ranged from one (elementary school not nished) to eight (academic degree).
f
The lower thresholds of the height and weight range seem rather extreme, but only four individuals reported a height below
150 cm (<0.1%), and two people scored their weight below 35 kg, and seven below 40 kg (0.1%).
Krieke et al. HowNutsAreTheDutch Crowdsourcing Study
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd. 135
There was considerable overlap between individuals
with severe levels on the three subscales: all individuals
who had severe symptom levels of stress and depression
also had severe symptom levels of anxiety. Gender differ-
ences were small (see Supplementary Material Table S3),
with women reporting slightly more symptoms of stress
(t= 9.89, p<0.001, d= 0.23), anxiety (t= 7.93, p<0.001,
d= 0.18), and depress ion than men ( t= 2.64, p<0.01,
d= 0.06). Finally, there were also many participants with-
out a single symptom of anxiety (26.6%), depression
(16.8%), or psychological stress (6.7%).
Most participants rated their quality of life fairly high (as
measured with the Manchester Short Assessment of quality
of life (MANSA), mean = 62, range 1284). In terms of hap-
piness the average rating was 6.9 on a scale from 0 to 10
(SD = 1.6, n= 10,152, median = 7.0. About 85% of the par-
ticipants rated their happiness a six or higher, and 40% an
eight or higher). Results of the Ryff total scale (range 46
234) indicate substantial individual differences in subjective
well-being. On average, women reported slightly lower well-
being than men (t=3.52, p<0.001, d= 0.08), mainly due
to less self-acceptance (t=4.59, p<0.001, d= 0.10) and
lower autonomy (t=21.77, p<0.001, d= 0.44). However,
compared to men, women reported more positive social re-
lationships (t=10.05, p<0.001, d= 0.21) and personal
growth (t= 7.77, p<0.001, d=0.17). Finally, even the
Figure 3. Heat map of the cross-sectional study participantsresidence versus population density map. Note: The population
density map on the right is derived from CBS statline and presents Dutch population densities per municipality in 2010 in
terms of number of inhabitants per square kilometre: From low in green (21250), via yellow (250500) and orange (light:
5001000, dark: 10002500) to red (25006000). The pictures show that the study coverage concurs very well with popula-
tion density scores.
Figure 4. Positive and negative affect in the cross-sectional
sample. A scatterplot with scores of the Positive and Nega-
tive Affect Schedule (PANAS). The black lines in the gure
indicate the mean values for positive affect (34.3) and neg-
ative affect (19.7). The darker the orange the higher the
number of people with that specic score. The black dots
represent actual observations.
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.136
subgroup of severely depressed, anxious, or stressed partici-
pants (about the highest 5% scores on the DASS, n=578)
rated their well-being fairly diverse (Ryff total score,
range = 46206, mean = 125, SD = 27, median = 123), just
as their general happiness (range = 010, mean = 4.1,
SD = 2.2, median = 4.0). Apparently, experiencing high
symptom levels does not necessarily preclude enjoying life.
Finally, we weighted our results for the selection bias fac-
tor. The results in Table 3 suggest that mood symptoms are
slightly underestimated in our sample, relative to the general
Dutch population. Anxiety was more prevalent than
depression/stress in both the HND and NEMESIS sample
(4.6%, past week, HND; and 12.4%, past 12 months, NEM-
ESIS). In the HND sample 3.9% reported severe depression
(past week), while 5.8% of the NEMESIS participants re-
ported a major depression in the past 12-months. In Figure 5
we present the prevalence of the nine symptoms of depres-
sion according to the DSM in the HND sample and in the
Lifelines sample, namely, depressed mood, diminished
interest, weight loss/gain, insomnia/hypersomnia, psycho-
motor agitation/retardation, fatigue or loss of energy,
worthlessness/guilt, concentration problems, and suicidal
ideation. The HND participants reported more symptoms
of depression (QIDS, self-report, past week) than the Life-
lines participants (MINI clinical interview, past year), but
these differences dissipate at the higher total scores, in line
with a lower threshold for self-report and timing effects.
Diary study
Sample characteristics
Up to 13 December 2014 629 participants completed the
diary study (5% of all HND participants). Of the diary
participants 532 (85%) also lled out additional cross-
sectional modules. The 629 participants were 517 women
(82%, mean age =39, SD = 13) and 112 men (18%, mean
age = 48, SD = 13), mainly Dutch (99%) and spread
throughout the Netherlands; ve participants were Belgian
and one had another nationality. Diary study participants
(n= 629) were on average 5.4 years younger than the other
HND participants (t= 9.78, p<0.001, d= 0.40), m ore of-
ten women (χ
2
= 84.51, p<0.001, d= 0.28), higher edu-
cated (t=5.67, p<0.001, d= 0.24), and they reported
lower well-being (t= 3.23, p<0.001, d= 0.14; see Supple-
mentary Material Tables S4 and S5). Details on the diary
study are presented elsewhere (Van der Krieke et al., sub-
mitted for publication).
Key results
Diary study participants completed 28,264 assessments in
total, with 45 assessments on average per participant
(range 090, SD =32). Since analyses were run for each
participant separately, group-based results are not pre-
sented here. About 48% (n= 302) participants completed
Figure 5. The prevalence of the nine DSM depression symptoms. The prevalence for each number of depression symptoms
in the HowNutsAreTheDuch (HND) sample and the Lifelines sample. The vertical axis shows the percentage of participants
with this particular number of symptoms.
Krieke et al. HowNutsAreTheDutch Crowdsourcing Study
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd. 137
enough assessments (t59) to receive basic feedback, and
38% (n= 238) completed enough assessment s (t68) to
receive additional advanced feedback, including a concur-
rent and dynamic personal network model.
Evaluation of the website
The evaluation questionnaire was completed by 3093 par-
ticipants who were on average 48 years old (SD = 14), and
65% were women. In the evaluation questionnaire partic-
ipants scored six components of the cross-sectional study
of the HND website on a scale from 1 to 10. In short,
the mean score for the lay-out of the website was rated
7.6, the cross-sectional modules 7.7, the presented results
7.3, and the overall judgement 7.7. The diary study was
evaluated separately, and these results are presented else-
where (Van der Krieke et al., submitted for publication).
Missing data
In HND we use digital questionnaires of which most are
programmed such that they need to be completed from
top to bottom and no items can be left blank. As a result,
the missing data within questionnaires is minimal. In the
cross-sectional study many questionnaire modules wereop-
tional, so not all participants completed all modules.
Supplementary Material Table S2 shows how many partici-
pants completed each module (range generally from 800 to
12,000). In the diary study, many participants missed as-
sessments, as we expected. To present reliable network
models, we only ran VAR analyses for participants who
completed >75% of the data (68 observations, n= 238).
Missing data of these participants were imputed using
Amelia-II imputation (http://cran.r-project.org/web/pack-
ages/Amelia/Amelia.pdf). Time series data with many
missing observations are not suitable for time series analysis,
but can be included in multilevel analyses and data-mining
procedures, which is what we intend to do in the future.
Comments
HND is primarily a scientic endeavour to explore ways in
which the classication of psychiatric symptoms can be
improved, via data-driven studies of how mental vulnera-
bilities and strengths are related and interact, including
humour, empathy, and self-acceptance, and by zooming
into dynamic interactions between health-related vari-
ables. Additionally, we aspire to contribute to the debate
about mental health in the Netherlands, aiming to reduce
the stigma associated with psychiatric diagnosis (Hinshaw
and Stier, 2008). An emphasis on the dimensional nature
of mental health and attention for individualsstrengths
may contribute to opportunities to increase mental health
in people suffering from mental symptoms.
Nearly half of the population will meet current DSM
criteria for a mental disorder at some time in their life, but
this does not mean that they will all need treatment (Kessler
et al., 2005). Health is best dened by the person (rather
than by the doctor), according to his or her functional
needs, which can be the meaning of personalized medi-
cine(Lancet, 2009; Perkins, 2001). Doctors, then, are part-
ners in delivering those needs. Diary approaches can play a
role in this process of empowerment, as they enable for per-
sonalized models that can shed light on etiology and per-
sonal dynamics, as well as personalized solutions, and
merit the perspective of health as peoples ability to adapt
to their environments and self-manage (Duckworth et al.,
2005; Huber et al., 2011; Solomon, 2014). Goals and criteria
for treatment successoften differ substantially between
clinicians and their patients, which may explain part of the
high drop-out rates (2560%) for most psychiatric interven-
tions (Perkins, 2001; Tehrani et al., 1996).
Mental symptoms may also be seen as more than de-
fects to be corrected, as individualsdifferences may be
their very strengths. For example, individuals with autism
may be great scientists, mathematicians, or software testers
(Mottron, 2011; Solomon, 2014), while anxious individ-
uals may be rather creative, sensitive, and agreeable, thus
perfect employees for social job tasks (George et al.,
2002; Stossel, 2013). Antagonistic, mistrustful, uncoopera-
tive and rude people, in contrast, may be excellent drill
sergeants or bill collectors (George and Jones, 2002, p. 50).
Furthermore, genes underlying creativity also increase vul-
nerability for schizophrenia and bipolar disorder (Power
et al., 2015). Everyone has both strengths and weaknesses,
but which is which is a function of the context in which
we live and grow (Darwin, 1859).
There can even be tension between identity and illness, as
some symptomscan feel self-congruent and come with-
out biographical breaks (e.g. in most personality disorders
and autistic people), while other symptoms seem to invade
an identity (e.g. in schizophrenia), and even harmoften
has a normative component (Dehue, 2014; Gutiérrez et al.,
2008; Solomon, 2014). A reconsideration of diversity and fo-
cus on individual strengths and resources thatmay compen-
sate for or buffer against the expression of mental
symptoms, may help people to preserve acceptable levels
of mental well-being despite the presence of psychopathol-
ogy (Harkness and Luther, 2001; Sheldon et al., 2011; Solo-
mon, 2014). This would t in with a concept of mental
health as a hybrid of absence of mental illness and the pres-
ence of well-being and mental resources (Duckworth et al.,
2005; Keyes, 2007).
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.138
In sum, our aim is to improve classication for re-
search purposes, and one promising direction is a dimen-
sional and dynamic approach that acknowledges the role
of the interactions between mental symptoms and
strengths (Duckworth et al., 2005). Although categoriza-
tion may be helpful in clinical practice to reach treatment
decisions, this procedure may be supplemented with a
patient-tailored treatment via the introduction of diary
studies and personalized models (Van der Krieke et al.,
submitted for publication).
Planned analyses
Data analyses of the HND project will be centred around
three sets of analyses. The rst set of analyses will focus
on identifying proles of mental symptoms and strengths
based on the cross-sectional data. For example, a study
of predictors for emotional and psychological well-being
across the lifespan (Jeronimus et al., submitted for publi-
cation), or of resources that enable people to preserve sub-
jective well-being despite severe mental symptoms (Bos
et al., in preparation). The second set of analyses will zoom
into the dynamic relationships between mood, cognitions,
and behaviours at the individual level, for which the diary
data will be used. To this end we will use time series anal-
ysis and multilevel analysis. A rst study showed that
prosocial behaviour and positive affect enhance one an-
other in daily life, and may thereby maintain positive men-
tal states (Snippe et al., submitted for publication). A third
set of analyses will apply machine learning techniques to
nd predictors for happiness (Wanders et al., in prepara-
tion), and to integrate dimensional and categorical ap-
proaches in the identication of affective subtypes
(Wardenaar et al., in preparation). Up to 1 July 2015, a to-
tal of 32 research proposals have been accepted by the
HND scientic board.
Dissemination of study ndings
Apart from publication in scientic journals, study nd-
ings are presented to our participants, via the HND
website. As described earlier, all participants receive indi-
vidual results via the website after completing a question-
naire module in the cross-sectional study, and after
completing the diary study. In addition, participants are
informed about study results via a newsletter (three to four
times a year). Study results are also communicated to the
broader Dutch population via news articles, radio inter-
views, podium discussions, and presentations open to the
general public.
Strengths and limitations of the HND project
One of the strengths of the HND project is that we involve
the general public in mental health research and in the de-
bate about how mental health should be conceptualized.
The project website provides participants with the opportu-
nity to gain insight into their mental health, whether or not
by comparing their scores to scores of other participants.
Moreover, the combination of (a) measuring mental symp-
toms and strengths and (b) our longitudinal time-intensive
design may allow for a more accurate and in-depth descrip-
tion of the dynamics of mental health and ill-health than
most studies are able to provide (Duckworth et al., 2005;
Keyes, 2007; Lamiell, 1998; Molenaar and Campbell, 2009;
Piantadosi et al., 1988). This broad range of assessed mental
strengths set HND apart from previous studies such as
NEMESIS and Lifelines.
The most salient limitation of our project is the prob-
lem of representativeness (self-selection bias), especially
the overrepresentation of highly educated strata and
women. To estimate the extent to which selection effects
curved our results we weighted our sample against the
proportions in the general Dutch population, and com-
pared the HND sample with the NEMESIS-2 and Lifelines
studies. Results suggest that scores of HND participants
are likely to deviate somewhat from population averages
on several psychological characteristics (mainly those asso-
ciated with differences in education), which might attenu-
ate the generalizability of our results (just as in NEMESIS
and Lifelines). For this reason NEMESIS weighted their re-
sults (de Graaf et al., 2010). Nevertheless, in the HND,
NEMESIS, and the Netherlands Study of Anxiety and
Stress (NESDA) anxiety was more prevalent than depres-
sion, and the small gender differences were even compara-
ble in size (e.g. for anxiety in HND d= 0.18 and NESDA
d= 0.11; for depression in HND d= 0.06 and NESDA
d= 0.08, see Jeronimus et al., 2013).
A large random sample from the general Dutch popu-
lation (without strong selection bias and non-response)
would require immense resources. Note that only 58.6%
of the random sample for NEMESIS-2 actually partici-
pated in that study (de Graaf et al., 2010). In Lifelines only
24.5% of the intended sample invited via their general
practitioner participated, while two-thirds of the assessed
sample resulted from self-selection via other means than
the general practitioner (see Scholtens et al., 2014).
It also remains doubtful whether a random sample
would have yielded knowledge about individual dynamics
that would be more applicable, informative, or transferable
(generalizable) at the personal level, see Molenaar and
Campbell (2009). Exactly therefore we implemented our
Krieke et al. HowNutsAreTheDutch Crowdsourcing Study
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd. 139
diary study. Moreover, we believe that the underlying facul-
ties of the mind (Panksepp and Biven, 2012) as well as the
structure in our data (Kendler and Parnas, 2015) will not
be different in subsamples of the population. Selection ef-
fects may thus, at worst, bias prevalence estimates (average
symptom counts), but we deem it unlikely that selection ef-
fects invalidate research into the associations and interac-
tions between personal vulnerabilities and resources.
Self-selection is not necessarily problematic, as previ-
ous crowdsourcing studies attracted more diverse partici-
pants than any other means of recruitment did (Gosling
et al., 2004; Revelle et al., 2010; Skitka and Sargis, 2006).
For example, HND sampled more participants above age
65 (9% versus 19% in the population) than Lifelines
(7.6%) and NEMESIS, which excluded people older than
64. This may reect that the Netherlands are among the
countries with most and fastest internet connections per
capita worldwide (90% of the households is connected).
Another limitation concerns the diary study. We allowed
our diary participants to complete their questionnaire until
one hour after the prompt. Methodologically, the presence
of this time window may have biased the results. For in-
stance, when participants received the prompt at a busy mo-
ment, they had the opportunity to postpone their response
to a more quiet moment, in which different emotions were
experienced and reported. However, our data indicated that
most diary questionnaires were completed within 12 mi-
nutes after the prompt (mean = 18.0, SD = 15.7), thus this
methodological bias is probably small.
Conclusion
The HND project has resulted in a rich dataset containing
both cross-sectional and intensive longitudinal data pro-
viding information about mental symptoms and strengths,
and their dynamic interactions. The data is used to provide
personalized feedback to participants about their mental
health, to study proles of mental symptoms and
strengths, and to zoom into the ne-grained level of dy-
namic relationships between variables over time.
Acknowledgements
The HowNutsAreTheDutch (HND) project is funded by a VICI
grant (no. 91812607) received by Peter de Jonge from the Neth-
erlands Organization for Scientic Research (NWO-ZonMW)
and by the University Medical Center Groningen Research
Award 2013, also received by Peter de Jonge. Part of the HND
project was realized in collaboration with the Espria Academy.
Espria is a health care group in the Netherlands consisting of
multiple companies targeted mainly at the elderly population.
The authors want to thank all participants of HND for their par-
ticipation and valuable contribution to this research project.
The authors also thank the RoQua team, Inge ten Vaarwerk, Es-
ther Hollander, Jasper van de Gronde, Joris Slaets, Ester Kuiper
and Chantal Bosman for their contributions.
Declaration of interest statement
The authors have no competing interests.
References
Abdel-Khalek A. (2006) Measuring happiness with a
single-item scale. Social Behavior & Personality:
An International Journal,34(2), 139149.
Arrindell W.A., Ettema J.H.M. (1986) SCL-90.
Handleiding bij een multidimensionale
psyhopathologie indicator,Lisse:Swets&Zeitlinger.
Baer R., Smith G., Hopkins J., Krietemeyer J., Toney
L. (2006) Using self-report assessment methods
to explore facets of mindfulness. Assessment,13
(1), 2745. DOI:10.1177/1073191105283504.
Baer R.A., Smith G.T., Lykins E., Button D.,
Krietemeyer J., Sauer S., Walsh E., Duggan
D., Williams J.M.G. (2008) Construct validity
of the ve facet mindfulness questionnaire in
meditating and nonmeditating samples. Assess-
ment,15(3), 329342, DOI: 10.1177/
1073191107313003
Barge-Schaapveld D.Q.C.M., Nicolson N.A.,
Berkhof J., deVries M.W. (1999) Quality of life
in depression: daily life determinants and vari-
ability. Psychiatry Research,88(3), 173189.
Baron-Cohen S., Wheelwright S. (2004) The em-
pathy quotient: an investigation of adults with
asperger syndrome or high functioning autism,
and normal sex differences. Journal of Autism
and Developmental Disorders,34(2), 163175.
DOI:10.1023/B:JADD.0000022607.19833.00.
Barrett L., Russell J. (1998) Independ ence and bipolar-
ity in the structure of current affect. Journal of Per-
sonality and Social Psychology,74(4), 967984.
Bevelander K.E., Kaipainen K., Swain R., Dohle S.,
Bongard J.C., Hines P.D.H., Wansink B. (2014)
Crowdsourcing novel childhood predictors of
adult obesity. PLoS ONE,9(2), e87756.
Blaauw F., van der Krieke L., Bos E., Emerencia A.
C., Jeronimus B.F., Schenk M., de Vos S.,
Wardenaar K., Wanders R., Wigman H., Aiello
M., de Jonge P. (2014) HowNutsAreTheDutch:
personalized feedback on a national scale. As-
sociation for the Advancement of Articial Intel-
ligence (AAAI) Fall Symposium Series, DOI:
10.13140/2.1.4203.1683
Bolger N., Davis A., Rafaeli E.(2003) Diary methods:
Capturing life as it i s lived. Annual Reviewof Psy-
chology,54,579616. DOI:10.1146/annurev.
psych.54.101601.145030.
Bos E.H., Snippe E., de Jonge P., Jeronimus B.F. (In
preparation) Preserving mental wellbeing in
the face of psychopathology: buffering effects
of personal strengths and resources.
Brabham D.C. (2008) Crowdsourcing as a
model for problem solving: an introduction
and cases. Convergence: The International
Journal of Research into New Media Technol-
ogies,14(1), 7590. DOI:10.1177/13548565
07084420.
Brandt P.T., Williams J.T. (2007) Multiple Time
Series Models, Thousand Oaks: CA, Sage
Publications.
Clark L.A., Watson D. (1991) Tripartite model of
anxiety and depression psychometric evi-
dence and taxonomic implications. Journal of
Abnormal Psychology,100(3), 316336.
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.140
Clark L.A., Watson D., Reynolds S. (1995) Diagnosis
and classication of psychopathology challenges
to the current system and future-directions. An-
nual Review of Psychology,46,121153.
DOI:10.1146/annurev.ps.46.020195.001005.
Collip I., Oorschot M., Thewissen V., Van Os J.,
Bentall R., Myin-Germeys I. (2011) Social
world interactions: how company connects to
paranoia. Psychological Medicine,41(5),
911921. DOI:10.1017/S0033291710001558.
Condon D.M., Revelle W. (2014) The international
cognitive ability resource: development and
initial validation of a public-domain measure.
Intelligence,43(0), 5264. DOI: 10.1016/j.
intell.2014.01.004
Csikszentmihalyi M., Larson R. (1987) Validity
and reliability of the experience-sampling
method. Journal of Nervous and Mental Dis-
ease,175(9), 526536. DOI:10.1097/
00005053-198709000-00004.
Darwin C. (1859) The Origin of Species, London,
Collectors Library: CRW Publishing Limited.
De Graaf R., ten Have M., van Dorsselaer S. (2010) The
Netherlands mental health survey and incidence
study-2 (NEMESIS-2): design and methods. Inter-
national Journal of Methods in Psychiatric Research,
19(3), 125141. DOI:10.1002/mpr.317.
Dean K., Stevens H., Mortensen P.B., Murray R.M.,
Walsh E., Pedersen C.B. (2010) Full spectrum
of psychiatric outcomes among offspring with
parental history of mental disorder. Archives
of General Psychiatry,67(8), 822829.
DOI:10.1001/archgenpsychiatry.2010.86.
De Beurs E., van Dyck R., Marquenie L.A., Lange
A., Blonk R.W.B. (2001) De DASS; een
vragenlijst voor het meten van depressie, angst
en stress. Gedragstherapie,34,3553.
De Fruyt F., Hoekstra H. (2014) NEO-PI-3
persoonlijkheidsvragenlijst,Amsterdam:Hogrefe.
De Klerk M., Van Yperen N.W., Postma S.,
Kamsma P.Y.J. (2003) Doen, voelen en
denken: drie gedragsstijlen toegepast in de
organisatieadviespraktijk. Gedrag En Organisatie,
3,169178.
Dehue T. (2014) Betere mensen, Amsterdam:
Uitgeverij Augustus.
Doane L.D., Adam E.K. (2010) Loneliness and cor-
tisol: momentary, day-to-day, and trait associ-
ations. Psychoneuroendocrinology,35(3),
430441. DOI:10.1016/j.psyneuen.2009.08.005.
Doré B., Ort L., Braverman O., Ochsner K.N. (2015)
Sadness shifts to anxiety over time and distance
from the national tragedy in Newtown, Con-
necticut. Psychological Science,26(4), 363373.
DOI:10.1177/0956797614562218.
Duckworth A.L., Steen T.A., Seligman M.E.P. (2005)
Positive psychology in clinical practice. Annual
Review of Clinical Psychology,1(1), 629651.
DOI:10.1146/annurev.clinpsy.1.102803.144154.
Dumont F. (2010) A History of Personality Psychol-
ogy: Theory, Science, and Research from Helle-
nism to the Twenty-rst Century, New York:
Cambridge University Press.
Durbin C.E., Hicks B.M. (2014) Personality and psy-
chopathology: a stagnant eld in need of develop-
ment. European Journal of Personality,28(4), 362.
Emerencia A.C., van der Krieke L., Bos E., de Jonge
P., Petkov N., Aiello M. (2015) Automating
vector autoregression on electronic patient di-
ary data. IEEE Journal of Biomedical and Health
Informatics. DOI:10.1109/JBHI.2015.2402280.
Fordyce M.W. (1988) A review of research on the
happiness measures: a sixty second index of
happiness and mental health. Social Indicators
Research,20(4), 355381.
George J.M., Jones G.R. (2002) Organizational Be-
havior, 3rd edn, NJ, Pearson: Upper Saddle
River.
Glaesmer H., Rief W., Martin A., Mewes R., Brähler
E., Zenger M., Hinz A. (2012) Psychometric
properties and population-based norms of the
life orientation test revised (LOT-R). British
Journal of Health Psychology,17(2), 432445.
DOI:10.1111/j.2044-8287.2011.02046.x.
Gosling S., Vazire S., Srivastava S., John O. (2004)
Should we trust web-based studies? A compara-
tive analysis of six preconceptions about internet
questionnaires. American Psychologist,59(2),
93104. DOI:10.1037/0003-066X.59.2.93.
Gutiérrez F., Navinés R., Navarro P., García-Esteve L.,
Subirá S., Torrens M., Martín-Santos R. (2008)
What do all personality disorders have in com-
mon? Ineffectiveness and uncooperativeness.
Comprehensive Psychiatry,49(6), 570578.
Hand E. (2010) People power. Nature,466(7307),
685687. DOI:10.1038/466685a.
Harkness K.L., Luther J. (2001) Clinical risk factors for
the generation of life events in major depression.
Journal of Abnormal Psychology,110(4), 564572.
Hinshaw S.P., Stier A. (2008) Stigma as related to
mental disorders. Annual Review of Clinical
Psychology,4, 367393. DOI:10.1146/annurev.
clinpsy.4.022007.141245.
Horwitz A.V., Wakeeld J.C. (2007) The Loss of
Sadness: How Psychiatry Transformed Normal
Sorrow into Depressive Disorder, New York:
Oxford University Press.
Howe J. (2006) Crowdsourcing: A denition,
http://www.crowdsourcing.typepad.com/cs/
2006/06/crowdsourcing_a.html [2 June 2006].
Huber M., Knottnerus J.A., Green L., Horst H.v.d.,
Jadad A.R., Kromhout D., Leonard B., Lorig
K., Loureiro M.I., van der Meer J.W.M.,
Schnabel P., Smith R., van Weel C., Smid H.
(2011) How should we dene health?, BMJ,
343, d4163. DOI: 10.1136/bmj.d4163
Hyman S.E. (2007) Can neuroscience be integrated
into the DSM-V? Nature Reviews. Neuroscience,
8(9), 725732.
Insel T. (2013) Transforming diagnosis, http://
www.nimh.nih.gov/about/director/2013/
transforming-diagnosis.shtm [29 April 2013].
Jeronimus B.F., van der Krieke L., Blaauw F.J.,
Schenk H.M.,Slaets J., Emerencia A.C.,de Jonge
P. (Submitted for publication) In search of
wisdom among the elderly: a study of subjec-
tive wellbeing across the lifespan (Study 1)
and the role of capabilities (Study 2) in the
Netherlands.
Jeronimus B.F., Ormel J., Aleman A., Penninx B.
W.J.H., Riese H. (2013) Negative and positive
life events predict small but long-term change
in neuroticism. Psychological Medicine,43(11),
24032415. DOI:10.1017/S0033291713000159.
Kapur S., Phillips A.G., Insel T.R. (2012) Why has
it taken so long for biological psychiatry to de-
velop clinical tests and what to do about it?
Molecular Psychiatry,17(12), 11741179.
Kendell R., Jablensky A. (2003) Distinguishing be-
tween the validity and utility of psychiatric di-
agnoses. American Journal of Psychiatry,
160(1), 412. DOI:10.1176/appi.ajp.160.1.4.
Kendler K.S. (1996) Major depression and general-
ised anxiety disorder same genes, (partly) dif-
ferent environments revisited. British Journal
of Psychiatry,168,6875.
KendlerK.S. (2008) Explanatory models for psychiat-
ric illness. American Journal of Psychiatry,165(6),
695702. DOI:10.1176/appi.ajp.2008.07071061.
Kendler K.S. (2012) The dappled nature of causes of
psychiatric illness: replacing the organic-
functional/hardware-software dichotomy with
empirically based pluralism. Molecular Psychia-
try,17(4), 377388. DOI:10.1038/mp.2011.182.
Kendler K.S., Firs tM .B. (2010)Altern ative futures for
the DSM revision process: iteration v. paradigm
shift. The British Journal of Psychiatry,197(4),
263265. DOI:10.1192/bjp.bp.109.076794.
Kendler K.S., Parnas J. (eds). (2015) Philosophical Issues
in Psychiatry III: The Nature and Sources of Histor-
ical Change,Oxford:OxfordUniversity Press.
Kendler K.S., Zachar P., Craver C. (2011) What
kinds of things are psychiatric disorders? Psy-
chological Medicine,41(6), 11431150.
DOI:10.1017/S0033291710001844.
Krieke et al. HowNutsAreTheDutch Crowdsourcing Study
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd. 141
Kessler R.C., Berglund P., Demler O., Jin R.,
Merikangas K.R., Walters E.E. (2005) Lifetime
prevalence and age-of-onset distributions of
DSM-IV disorders in the national comorbidity
survey replication. Archives of General Psychiatry,
62(6), 593602. DOI:10.1001/archpsyc.62.6.593.
Keyes C.L.M. (2007) Promoting and protecting mental
health as ourishing: a complementary strategy for
improving national mental health. American Psy-
chologist,62(2), 95108. DOI:10.1037/0003-
066X.62.2.95.
Klimstra T.A., Sijtsema J.J., Henrichs J., Cima M.
(2014) The dark triad of personality in adoles-
cence: psychometric properties of a concise
measure and associations with adolescent ad-
justment from a multi-informant perspective.
Journal of Research in Personality,53(0),
8492. DOI:10.1016/j.jrp.2014.09.001.
Konings M., Bak M., Hanssen M., Van Os J.,
Krabbendam L. (2006) Validity and reliability
of the CAPE: a self-report instrument for the
measurement of psychotic experiences in the
general population. Acta Psychiatrica
Scandinavica,114(1), 5561. DOI:10.1111/
j.1600-0447.2005.00741.x.
Kroenke K., Spitzer R.L., Williams J.B. (2002) The
PHQ-15: validity of a new measure for evaluat-
ing the severity of somatic symptoms. Psycho-
somatic Medicine,64(2), 258266.
Krueger R.F. (1999) The structure of common
mental disorders. Archives of General Psychia-
try,56(10), 921926. DOI:10.1001/archpsyc.
56.10.921.
Krueger R.F., Markon K.E. (2006) Reinterpreting
comorbidity: a model-based approach to un-
derstanding and classifying psychopathology.
Annual Review of Clinical Psychology,2, 111.
DOI:10.1146/annurev.clinpsy.2.022305.
095213.
Kupfer D.J.F., First M.B., Regier D.A. (2002) ARe-
search Agenda for DSM-V, Arlington, VA:
American Psychiatric Association.
Lamiell J.T. (1998) `Nomotheticand `Idiographic:
contrasting windelbands understanding with
contemporary usage. Theory & Psychology,8(1),
2338. DOI:10.1177/0959354398081002.
Lancet T. (2009) What is health? The ability
to adapt. The Lancet,373(9666), 781.
DOI:10.1016/S0140-6736(09)60456-6.
Larson J.S. (1999) The conceptualization of health.
Medical Care Research and Review,56(2),
123136. DOI:10.1177/107755879905600201.
Lovibond P.F., Lovibond S.H. (1995a) The structure
of negative emotional states comparison of the
depression anxiety stress scales (DASS) with the
beck depression and anxiety inventories. Behav-
iour Research and Therapy,33(3), 335343.
DOI:10.1016/0005-7967(94)00075-U.
Lovibond S.H., Lovibond P.F. (1995b) Manual for
the depression Anxiety Stress Scales (2nd edi-
tion), Sydney: Psychology Foundation.
Lumley T. (2014) Analysis of complex survey sam-
ples. Journal of Statistical Software,9(8), 115.
Luthar S.S., Cicchetti D., Becker B. (2000) The con-
struct of resilience: a critical evaluation and
guidelines for future work. Child Development,
71(3), 543562. DOI:10.1111/1467-8624.00164.
Lütkepohl H. (2006) Structural vector autoregressive
analysis for cointegrated variables. In Hübler O.,
Frohn J. (eds) Modern Econometric Analysis,
pp. 7386, Berlin: Springer, DOI: 10.1007/3-
540-32693-6_6
Martin R.A., Puhlik-Doris P., Larsen G., Gray J., Weir
K. (2003) Individual differences in uses of humor
and their relation to psychological well-being: de-
velopment of the humor styles questionnaire.
Journal of Research in Personality,37(1), 4875.
Mineka S., Watson D., Clark L. (1998) Comorbid-
ity of anxiety and unipolar mood disorders.
Annual Review of Psychology,49, 377412.
DOI:10.1146/annurev.psych.49.1.377.
Molenaar P.C.M., Campbell C.G. (2009) The new
person-specic paradigm in psychology. Current
Directions in Psychological Science,18(2),
112117. DOI:10.1111/j.1467-8721.2009.01619.x.
Monk T.H., Reynolds C.F., Kupfer D.J., Buysse D.J.,
Coble P.A., Hayes A.J., Machen M.A., Petrie S.
R., Ritenour A.M. (1994) The Pittsburgh sleep
diary. Journal of Sleep Research,3(2), 111120.
Mottron L. (2011) Changing perceptions: the
power of autism. Nature,479(7371), 3335.
Myin-Germeys I., Oorschot M., Collip D., Lataster
J., Delespaul P., van Os J. (2009) Experience
sampling research in psychopathology: open-
ing the black box of daily life. Psychological
Medicine,39(09), 15331547. DOI:10.1017/
S0033291708004947.
Nieboer A., Lindenberg S., Boomsma A., Van
Bruggen A. (2005) Dimensions of well-being
and their measurement: the SPF-IL scale. So-
cial Indicators Research,73(3), 313353.
DOI:10.1007/s11205-004-0988-2.
Ooms J. (2014) The OpenCPU system: Towards a
universal interface for scientic computing
through separation of concerns, ArXiv e-Prints,
1406.4806 [accessed 1 March 2015]
Ormel J., Jeronimus B.F., Kotov R., Riese H.,
Bos E.E., Hankin B., Rosmalen J.G.M.
(2013) Neuroticism and common mental
disorders : meaning and ut ility of a co mplex
relationship. Clinical Psychology Review,33(5),
686697.
Panksepp J., Biven L. (2012) The Archaeology of
Mind: Neuroevolutionary Origins of Human
Emotions, New York: Norton & Company.
Paulhus D.L., Williams K.M. (2002) The dark triad
of personality: Narcissism, machiavellianism,
and psychopathy. Journal of Research in Person-
ality,36(6), 556563. DOI: http://dx.doi.org/
10.1016/S0092-6566(02)00505-6.
Peeters F.P.M.L., Ponds R.H.W.M., Vermeeren M.
T.G. (1996) Affectiviteit en zelfbeoordeling van
depressie en angst. Tijdschrift Voor De
Psychiatrie,38(3), 240250.
Perkins R. (2001) What constitutes success? The
British Journal of Psychiatry,179(1), 910.
DOI:10.1192/bjp.179.1.9.
Piantadosi S., Byar D.P., Green S.B. (1988) The
ecological fallacy. American Journal of Epidemi-
ology,127(5), 893904.
Priebe S., Huxley P., Knight S., Evans S. (1999) Ap-
plication and results of the Manchester short
assessment of quality of life (MANSA). Inter-
national Journal of Social Psychiatry,45(1),
712. DOI:10.1177/002076409904500102.
Priebe S., Reininghaus U., McCabe R., Burns T.,
Eklund M., Hansson L., Junghan U., Kallert
T., van Nieuwenhuizen C., Ruggeri M., Slade
M., Wang D. (2010) Factors inuencing sub-
jective quality of life in patients with schizo-
phrenia and other mental disorders: a pooled
analysis. Schizophrenia Research,121(13),
251258. DOI: 10.1016/j.schres.2009.12.020
Power R.A., Steinberg S., Bjornsdottir G., Rietveld
C.A., Abdellaoui A., Nivard M.M.,
Johannesson, M., Galesloot, T.E., Hottenga,
J.J., Willemsen, G., Cesarini, D., Benjamin, D.
J., Magnusson, P.K.E., Ullen, F., Tiemeier, H.,
Hofman, A., van Rooij, F.J.A., Walters, G.B.,
Sigurdsson, E., Thorgeirsson, T.E., Ingason,
A., Helgason, A., Kong, A., Kiemeney, L.A.,
Koellinger, P., Boomsma, D.I., Gudbjartsson,
D., Stefansson, H., Stefansson, K. (2015) Poly-
genic risk scores for schizophrenia and bipolar
disorder predict creativity. Nature Neuroscience,
18(7), 953955. DOI:10.1038/nn.4040.
Psychiatric Genomics Consortium (2015) Psychiat-
ric genome-wide association study analyses
implicate neuronal, immune and histone path-
ways. Nature Neuroscience,18(2), 199209.
Raes F., Daems K., Feldman G.C., Johnson S.L.,
Van Gucht D. (2009) A psychometric evalua-
tion of the Dutch version of the responses to
positive affect questionnaire. Psychologica
Belgica,49(4), 293310.
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.142
Revelle W., Wilt J., Rosenthal A. (2010) Individual
differences in cognition: new methods for ex-
amining the personality-cognition link. In
Gruszka A., Matthews G., Szymura B. (eds)
Handbook of Individual Differences in Cogni-
tion, New York: Springer.
Roest A.M., de Jonge P., Williams C.D., de Vries Y.
A., Schoevers R.A., Turner E.H. (2015)
Reporting bias in clinical trials investigating
the efcacy of second generation antidepres-
sants in the treatment of anxiety
disorders. JAMA Psychiatry,72(5), 500510.
DOI:10.1001/jamapsychiatry.2015.15.
Rush A., Bernstein I., Trivedi M., Carmody T.,
Wisniewski S., Mundt J., Shores-Wilson K.,
Biggs M., Woo A., Nierenberg A., Fava M.
(2006) An evaluation of the quick inventory
of depressive symptomatology and the Ham-
ilton rating scale for depression: a sequenced
treatment alternatives to relieve depression
trial report. Biological Psychiatry,59(6),
493501, DOI: 10.1016/j.biopsych.2005.
08.022
Rush A., Trivedi M., Ibrahim H., Carmody T.,
Arnow B., Klein D., Markowitz J., Ninan P.,
Kornstein S., Manber R., Thase M., Kocsis J.,
Keller M. (2003) The 16-item quick inventory
of depressive symptomatology (QIDS), clini-
cian rating (QIDS-C), and self-report (QIDS-
SR): a psychometric evaluation in patients with
chronic major depression. Biological Psychiatry,
54(5), 573583, DOI: 10.1016/S0006-3223(02)
01866-8
Rush A.J., Gullion C.M., Basco M.R., Jarrett R.B.,
Trivedi M.H. (1996) The inventory of depressive
symptomatology (IDS): psychometric proper-
ties. Psychological Medicine,26(3), 477486.
Savin-Williams R.C., Demo D.H. (1983) Situa-
tional and transituational determinants of ado-
lescent self-feelings. Journal of Personality and
Social Psychology,44(4), 824833.
DOI:10.1037/0022-3514.44.4.824.
Scholtens S., Smidt N., Swertz M.A., Bakker S.J.,
Dotinga A., Vonk J.M., van Dijk F., van Zon
S.K., Wijmenga C., Wolffenbuttel B.H., Stolk
R.P. (2014) Cohort prole: LifeLines, a three-
generation cohort study and biobank. Interna-
tional Journal of Epidemiology, DOI: 10.1093/
ije/dyu229
Seery M.D., Holman E.A., Silver R.C. (2010) Whatever
does not kill us: cumulative lifetime adversity, vul-
nerability, and resilience. Journal of Personality and
Social Psychology,99(6), 10251041.
Seligman M.E.P., Csikszentmihalyi M. (2000) Pos-
itive psychology: an introduction. American
Psychologist,55(1), 514. DOI:10.1037/0003-
066X.55.1.5.
Sheldon K.M., Kashdan T.B., Steger M.F. (eds).
(2011) Designing Positive Psychology: Taking
Stock and Moving Forward, New York: Oxford
University Press.
Skitka L.J., Sargis E.G. (2006) The internet as psy-
chological laboratory. Annual Review of Psy-
chology,57(1), 529555. DOI:10.1146/
annurev.psych.57.102904.190048.
Speckens A.E.M., Spinhoven P., Sloekers P.P.A.,
Bolk J.H., van Hemert A.M. (1996) A valida-
tion study of the Whitely Index, the illness at-
titude scales, and the somatosensory
amplication scale in general medical and gen-
eral practice patients. Journal of Psychosomatic
Research,40(1), 95104.
Snippe E., Jeronimus B.F., aan het Rot M., Bos E.
H., de Jonge P., Wichers M. (Submitted for
publication) Prosocial people smile: the daily
dynamics of an upward spiral.
Solomon A. (2014) Far from the Tree, London: Vin-
tage Books.
Stossel S. (2013) My Age of Anxiety: Fear, Hope,
Dread, and the Search for Peace of Mind, New
York: Borzoi Books.
Tehrani E., Krussel J., Borg L., Munk-Jorgensen P.
(1996) Dropping out of psychiatric treatment:
a prospective study of a rst-admission cohort.
Acta Psychiatrica Scandinavica,94(4), 266271.
DOI:10.1111/j.1600-0447.1996.tb09859.x.
Thombs B.D., Bernstein D.P., Lobbestael J., Arntz A.
(2009) A val idation s tudy of the D utch child-
hood trauma questionnaire-short form: factor
structure, reliability, and known-groups validity.
Child Abuse & Neglect,33(8), 518523.
Van der Krieke L., Emerencia A.C., Bos E.H.,
Rosmalen J.G., Riese H., Aiello M., Sytema S.,
de Jonge P. (2015) Ecological momentary as-
sessments and automated time series analysis
to promote tailored health care: A proof-of-
principle study. JMIR Research Protocols,
4(3), e100. DOI:10.2196/resprot.4000. PMID:
26254160.
Van der Krieke L., Blaauw F.J., Emerencia A.C.,
Schenk H.M., Bos E.H., de Jonge P., Jeronimus
B.F. (submitted for publication) Temporal dy-
namics of health and wellbeing in the Dutch
population: Nationwide access to momentary
assessments and automated generation of per-
sonalized feedback.
Van Dierendonck D. (2004) The construct validity of
Ryffs scales of psychological well-being and its
extension with spiritual well-being. Personality
and Individual Differences,36(3), 629643.
Veenhoven R. (1994) Is happiness a trait? Tests of
the theory that a better society does not make
people any happier. Social Indicators Research,
32(2), 101160.
Wakeeld J.C. (1992) The concept of mental dis-
order: on the boundary between biological
facts and social values. American Psychologist,
47(3), 373388. DOI:10.1037/0003-066X.47.
3.373.
Wanders R.B.K., Kingma E.M., Jeronimus B.F., de
Jonge P. (In preparation) Happiness lies in
the interaction: using LASSO to catch positive
and negative predictors in a large Dutch
crowdsourced study.
Wardenaar K., de Jonge P. (2013) Diagnostic het-
erogeneity in psychiatry: towards an empirical
solution. BMC Medicine,11(1), 201.
Wardenaar K.J., Wanders R.B.K., Jeronimus B.F.,
Wigman J.T.W., de Jonge P. (in preparation) In-
tegrating dimensional and categorical approaches
in the identication of affective subtypes.
Westerhof G., Keyes C. (2010) Mental illness and
mental health: the two continua model across
the lifespan. Journal of Adult Development,
17(2), 110119. DOI:10.1007/s10804-009-
9082-y.
Whooley O. (2014) Nosological reections: the
failure of DSM-5, the emergence of RDoC,
and the decontextualization of mental distress.
Society and Mental Health,4(2), 92110.
DOI:10.1177/2156869313519114.
Wichers M., Peeters F., Rutten B.P.F., Jacobs N.,
Derom C., Thiery E., Delespaul P., van Os J.
(2011) A time-lagged momentary assessment
study on daily life physical activity and affect.
Health Psychology,31(2), 135144, DOI:
10.1037/ a0025688
Widiger T.A., Samuel D.B. (2005) Diagnostic cat-
egories or dimensions? A question for the Di-
agnostic and Statistical Manual of Mental
Disorders Fifth Edition.Journal of Abnormal
Psychology,114(4), 494504. DOI:10.1037/
0021-843X.114.4.494.
Widiger T.A., Sankis L.M. (2000) Adult psychopa-
thology: issues and controversies. Annual
Review of Psychology,51(1), 377404.
DOI:10.1146/annurev.psych.51.1.377.
Wittkampf K., van Ravesteijn H., Baas K., van de
Hoogen H., Schene A., Bindels P., Lucassen
P., van de Lisdonk E., van Weert H. (2009)
The accuracy of Patient Health Questionnaire-
9 in detecting depression and measuring de-
pression severity in high-risk groups in primary
care. General Hospital Psychiatry,31(5),
451459.
Krieke et al. HowNutsAreTheDutch Crowdsourcing Study
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd. 143
World Health Organization (1946) Constitution of
the world-health-organization. Public Health
Reports,61(35), 12681277.
Yik M.S.M., Russell J.A., Barrett L.F. (1999) Struc-
ture of self-reported current affect: Integration
and beyond. Journal of Personality and Social
Psychology,77(3), 600619. DOI:10.1037/
0022-3514.77.3.600.
Supporting information
Additional supporting information may be found in the online version of this article at the publisher's web site.
HowNutsAreTheDutch Crowdsourcing Study Krieke et al.
Int. J. Methods Psychiatr. Res. 25(2): 123144 (2016). DOI: 10.1002/mpr
Copyright © 2015 John Wiley & Sons, Ltd.144
... In terms of the general content of questions, the time window that the questions refer to impacts the extent to which EMA can be conceptualized as a representative subset of the experiencing self. In many designs, people are asked about their experiences in the exact moment which as mentioned earlier, seems closely related to the experiencing self (e.g., "how sad do you feel right now?"; c.f., Thomas and Diener, 1990;Parkinson et al., 1995;Barrett, 1997;Ebner-Priemer et al., 2006;Ben-Zeev et al., 2009Ben-Zeev and Young, 2010;Dockray et al., 2010;Burns et al., 2011;Bylsma et al., 2011;Wenze et al., 2012;Kramer et al., 2014;van der Krieke et al., 2016;Kroeze et al., 2017;Lay et al., 2017;Colombo et al., 2019Colombo et al., , 2020Neubauer et al., 2020;Lucas et al., 2021;Ornée et al., 2021). An important drawback to this procedure however, is that there is a high chance that salient experiences will be missed. ...
... For example, such analysis of EMA/ERA data may result in the finding that for a person, the experience of feeling anxious at one point in time predicts the experience of feeling down at a later point in time . Some authors have suggested that these types of network models may help guide interventions (Fisher and Boswell, 2016;Borsboom, 2017;Fernandez et al., 2017;Fisher et al., 2019;Hofmann and Hayes, 2019), 2 and the first empirical efforts that have provided personalized feedback based on network analyzes have started to arise (van der Krieke et al., 2016;Kroeze et al., 2017;van Roekel et al., 2017;Epskamp et al., 2018). Kroeze et al. (2017), for example, analyzed the network of a patient suffering from treatment resistant anxious and depressive symptoms. ...
... As such, any possible increase in insight is currently often left in the open. Specifically, in many cases, the personalized feedback procedure lacks both a pre-and a post-measure of insight (c.f., Burns et al., 2011;Kramer et al., 2014;Hartmann et al., 2015;Simons et al., 2015;Snippe et al., 2016;van der Krieke et al., 2016;van Roekel et al., 2017;Ornée et al., 2021). Instead, in some of these studies, alleviation of symptoms is used as a post-measure (Kramer et al., 2014;Hartmann et al., 2015;Simons et al., 2015;Snippe et al., 2016;van Roekel et al., 2017). ...
Article
Full-text available
Ecological Momentary Assessment (EMA) in which participants report on their moment-to-moment experiences in their natural environment, is a hot topic. An emerging field in clinical psychology based on either EMA, or what we term Ecological Retrospective Assessment (ERA) as it requires retrospectivity, is the field of personalized feedback. In this field, EMA/ERA-data-driven summaries are presented to participants with the goal of promoting their insight in their experiences. Underlying this procedure are some fundamental assumptions about (i) the relation between true moment-to-moment experiences and retrospective evaluations of those experiences, (ii) the translation of these experiences and evaluations to different types of data, (iii) the comparison of these different types of data, and (iv) the impact of a summary of moment-to-moment experiences on retrospective evaluations of those experiences. We argue that these assumptions deserve further exploration, in order to create a strong evidence-based foundation for the personalized feedback procedure.
... Items scores are summed to yield a total score ranging from 14 to 98, with higher scores indicating greater perceived resilience. Total scores are categorized as very low , low (65)(66)(67)(68)(69)(70)(71)(72)(73), moderate (74)(75)(76)(77)(78)(79)(80)(81), moderately high (82-90), and high (91-98) [59]. ...
... If the MSSD is 50 or less, the variable is not included in the AutoVAR analyses. This threshold is used to ensure sufficient variability within each variable and, as such, increases the probability of finding a valid VAR model [67]. A model is considered valid if it passes all the necessary assumptions of a VAR analysis (i.e., stationarity, white noise, homoscedasticity, normality). ...
Article
Full-text available
Background Fear of cancer recurrence, depressive symptoms, and cancer-related fatigue are prevalent symptoms among cancer survivors, adversely affecting patients’ quality of life and daily functioning. Effect sizes of interventions targeting these symptoms are mostly small to medium. Personalizing treatment is assumed to improve efficacy. However, thus far the empirical support for this approach is lacking. The aim of this study is to investigate if systematically personalized cognitive behavioral therapy is more efficacious than standard cognitive behavioral therapy in cancer survivors with moderate to severe fear of cancer recurrence, depressive symptoms, and/or cancer-related fatigue. Methods The study is designed as a non-blinded, multicenter randomized controlled trial with two treatment arms (ratio 1:1): (a) systematically personalized cognitive behavioral therapy and (b) standard cognitive behavioral therapy. In the standard treatment arm, patients receive an evidence-based diagnosis-specific treatment protocol for fear of cancer recurrence, depressive symptoms, or cancer-related fatigue. In the second arm, treatment is personalized on four dimensions: (a) the allocation of treatment modules based on ecological momentary assessments, (b) treatment delivery, (c) patients’ needs regarding the symptom for which they want to receive treatment, and (d) treatment duration. In total, 190 cancer survivors who experience one or more of the targeted symptoms and ended their medical treatment with curative intent at least 6 months to a maximum of 5 years ago will be included. Primary outcome is limitations in daily functioning. Secondary outcomes are level of fear of cancer recurrence, depressive symptoms, fatigue severity, quality of life, goal attainment, therapist time, and drop-out rates. Participants are assessed at baseline (T0), and after 6 months (T1) and 12 months (T2). Discussion To our knowledge, this is the first randomized controlled trial comparing the efficacy of personalized cognitive behavioral therapy to standard cognitive behavioral therapy in cancer survivors. The study has several innovative characteristics, among which is the personalization of interventions on several dimensions. If proven effective, the results of this study provide a first step in developing an evidence-based framework for personalizing therapies in a systematic and replicable way. Trial registration The Dutch Trial Register (NTR) NL7481 (NTR7723). Registered on 24 January 2019.
... To present a real data application, we analyzed data from the national crowdsourcing study HowNutsAreTheDutch (Dutch: HoeGekIsNL; Krieke et al., 2016;Krieke et al., 2017), which started in May 2014. The HowNutsAreTheDutch (HND) project aims to study the mental health of the Dutch population by considering the mental health as "a dimensional and dynamic phenomenon" (Krieke et al., 2016, p. 124 Russell, 1998, andYik, Russell, andBarrett, 1999, as cited in Krieke et al., 2016). ...
... To present a real data application, we analyzed data from the national crowdsourcing study HowNutsAreTheDutch (Dutch: HoeGekIsNL; Krieke et al., 2016;Krieke et al., 2017), which started in May 2014. The HowNutsAreTheDutch (HND) project aims to study the mental health of the Dutch population by considering the mental health as "a dimensional and dynamic phenomenon" (Krieke et al., 2016, p. 124 Russell, 1998, andYik, Russell, andBarrett, 1999, as cited in Krieke et al., 2016). In a nutshell, the circumplex model assumes that affect can be explained by two dimensions: Valence (positive or negative) and activation. ...
Article
Full-text available
Traditionally, researchers have used time series and multilevel models to analyze intensive longitudinal data. However, these models do not directly address traits and states which conceptualize the stability and variability implicit in longitudinal research, and they do not explicitly take into account measurement error. An alternative to overcome these drawbacks is to consider structural equation models (state-trait SEMs) for longitudinal data that represent traits and states as latent variables. Most of these models are encompassed in the latent state-trait (LST) theory. These state-trait SEMs can be problematic when the number of measurement occasions increases. As they require the data to be in wide format, these models quickly become overparameterized and lead to nonconvergence issues. For these reasons, multilevel versions of state-trait SEMs have been proposed, which require the data in long format. To study how suitable state-trait SEMs are for intensive longitudinal data, we carried out a simulation study. We compared the traditional single level to the multilevel version of three state-trait SEMs. The selected models were the multistate-singletrait (MSST) model, the common and unique trait-state (CUTS) model, and the trait-state-occasion (TSO) model. Furthermore, we also included an empirical application. Our results indicated that the TSO model performed best in both the simulated and the empirical data. To conclude, we highlight the usefulness of state-trait SEMs to study the psychometric properties of the questionnaires used in intensive longitudinal data. Yet, these models still have multiple limitations, some of which might be overcome by extending them to more general frameworks. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
... A transition was therefore defined as pertaining to momentary mood, symptoms, sleep, and activities. These items were based on previous EMA research (31)(32)(33) and interviews with three patients and a psychiatrist on relevant constructs for people with BD. For the calculation of EWS, we selected the 17 EMA a Although N=9 patients had BD type II, suggesting they experience hypomanic transitions, we refer to both hypomanic and manic transitions as 'manic transitions' for consistency. ...
Article
Full-text available
Introduction For patients with bipolar disorder, early recognition of impending mood episodes is crucial to enable timely intervention. Longitudinal digital mood monitoring using ecological momentary assessment (EMA) enable prospective study of early warning signals (EWS) in momentary affective estates prior to symptom transitions. Objectives The present study examined in a unique longitudinal EMA data set whether EWS prospectively signal transitions to manic or depressive episodes. Methods Twenty bipolar type I/II patients completed EMA questionnaires five times a day for four months (average 491 observations per person), as well as weekly symptom questionnaires concerning depressive (Quick Inventory for Depressive Symptomatology) and manic (Altman Self-Rating Mania Scale) symptoms. Weekly data was used to determine transitions (i.e., abrupt increase in symptoms). Prior to these transitions, EWS (autocorrelation at lag-1 and standard deviation) were calculated in moving windows over 17 affective EMA states. Kendall’s tau was calculated to detect significant rises in the EWS indicator prior to the transition. Results Eleven patients reported one or two transitions to a mood episode. All transitions were preceded by at least one EWS. Average sensitivity for detecting EWS was slightly higher for manic episodes (36%) than for depressive episodes (25%). For manic episodes, EWS in thoughts racing, being full of ideas, and feeling agitated showed the highest sensitivity and specificity, whereas for depression, only feeling tired showed high sensitivity and specify. Conclusions EWS show promise in anticipating transitions to mood episodes in bipolar disorder. Further investigation is warranted. Disclosure No significant relationships.
Article
Background In bipolar disorder treatment, accurate episode prediction is paramount but remains difficult. A novel idiographic approach to prediction is to monitor generic early warning signals (EWS), which may manifest in symptom dynamics. EWS could thus form personalized alerts in clinical care. The present study investigated whether EWS can anticipate manic and depressive transitions in individual patients with bipolar disorder. Methods Twenty bipolar type I/II patients (with ≥ 2 episodes in the previous year) participated in ecological momentary assessment (EMA), completing five questionnaires a day for four months ( Mean = 491 observations per person). Transitions were determined by weekly completed questionnaires on depressive (Quick Inventory for Depressive Symptomatology Self-Report) and manic (Altman Self-Rating Mania Scale) symptoms. EWS (rises in autocorrelation at lag-1 and standard deviation) were calculated in moving windows over 17 affective and symptomatic EMA states. Positive and negative predictive values were calculated to determine clinical utility. Results Eleven patients reported 1–2 transitions. The presence of EWS increased the probability of impending depressive and manic transitions from 32-36% to 46–48% (autocorrelation) and 29–41% (standard deviation). However, the absence of EWS could not be taken as a sign that no transition would occur in the near future. The momentary states that indicated nearby transitions most accurately (predictive values: 65–100%) were full of ideas, worry, and agitation. Large individual differences in the utility of EWS were found. Conclusions EWS show theoretical promise in anticipating manic and depressive transitions in bipolar disorder, but the level of false positives and negatives, as well as the heterogeneity within and between individuals and preprocessing methods currently limit clinical utility.
Preprint
Full-text available
The unique and synergistic effects of daily risk and protective factors that shape our moods remain largely unknown because each is typically studied in isolation. Using experience sampling techniques 1396 Dutch adults reported on their positive and negative affect (PA/NA), social contact, emotion coping, physical activity, sleep quality, and negative events; thrice daily for 30 days (90 assessments). These five risk/protective factors combined explained approximately 15% of the variation in PA and 23% in NA, with emotion coping and sleep quality as the strongest predictors. All risk and protective factors influenced subsequent mental health, but examined collectively, only coping ability and sleep quality significantly affected individuals’ mental health. These results identify coping and sleep as the most interesting targets for depression interventions and psychotherapy to enhance positive mood and decrease negative mood.
Article
Full-text available
The trait-state-occasion model (TSO) is a popular model within the latent state-trait theory (LST). The TSO allows distinguishing the trait and the state components of the psychological constructs measured in longitudinal data, while also taking into account the carry-over effects between consecutive measurements. In the present study, we extend a multilevel version of the TSO model to allow for the combination of fixed and random situations, namely the mixed-effects TSO (ME-TSO). Hence, the ME-TSO model is a measurement model suitable to analyze intensive longitudinal data that allows studying the psychometric properties of the indicators per individual, the heterogeneity of psychological dynamics, and the person–situation interaction effects. We showcase how to use the model by analyzing the items of positive affect activation of the crowdsourcing study HowNutsAreTheDutch (HoeGekisNL).
Article
Full-text available
Introduction A common approach to personalizing psychological interventions is the allocation of treatment modules to individual patients based on cut-off scores on questionnaires, which are mostly based on group studies. However, this way, intraindividual variation and temporal dynamics are not taken into account. Automated individual time series analyses are a possible solution, since these can identify the factors influencing the targeted symptom in a specific individual, and associated modules can be allocated accordingly. The aim of this study was to illustrate how automated individual time series analyses can be applied to personalize cognitive behavioral therapy for cancer-related fatigue in cancer survivors and how this procedure differs from allocating modules based on questionnaires. Methods This study was a case report series (n = 3). Patients completed ecological momentary assessments at the start of therapy, and after three treatment modules (approximately 14 weeks). Assessments were analyzed with AutoVAR, an R package that automates the process of finding optimal vector autoregressive models. The results informed the treatment plan. Results Three cases were described. From the ecological momentary assessments and automated time series analyses three individual treatment plans were constructed, in which the most important predictor for cancer-related fatigue was treated first. For two patients, this led to the treatment ending after the follow-up ecological momentary assessments. One patient continued treatment until six months, the standard treatment time in regular treatment. All three treatment plans differed from the treatment plans informed by questionnaire scores. Discussion This study is one of the first to apply time series analyses in systematically personalizing psychological treatment. An important strength of this approach is that it can be used for every modular cognitive behavioral intervention where each treatment module addresses specific maintaining factors. If personalized CBT is more efficacious than standard, non-personalized CBT remains to be determined in controlled studies comparing it to usual care.
Article
Precision psychiatry stands to benefit from the latest digital technologies for assessment and analyses to tailor treatment towards individuals. Insights into dynamic psychological processes as they unfold in humans' everyday life can critically add value in understanding symptomatology and environmental stressors to provide individualized treatment where and when needed. Towards this goal, ambulatory assessment encompasses methodological approaches to investigate behavioral, physiological, and biological processes in humans' everyday life. It combines repeated assessments of symptomatology over time, e.g., via Ecological Momentary Assessment (e.g., smartphone-diaries), with monitoring of physical behavior, environmental characteristics (such as geolocations, social interactions) and physiological function via sensors, e.g., mobile accelerometers, global-positioning-systems, and electrocardiography. In this review, we expand on promises of ambulatory assessment in the investigation of mental states (e.g., real-life, dynamical and contextual perspective), on chances for precision psychiatry such as the prediction of courses of psychiatric disorders, detection of tipping points and critical windows of relapse, and treatment effects as exemplified by ongoing projects, and on future avenues of how ambulatory interventions can benefit personalized care for psychiatric patients (e.g., through real-time feedback in everyday life). Ambulatory assessment is a key contributor to precision psychiatry, opening up promising avenues in research, diagnoses, prevention and treatment.
Article
Full-text available
Objective: Recent developments in research and mobile health enable for a quantitative idiographic approach in health research. The present study investigates the potential of an electronic diary crowdsourcing study in the Netherlands for (1) large-scale automated self-assessment for individual-based health promotion and (2) enabling research at both the between-persons and within-persons level. To illustrate the latter, we examined between-persons and within-persons associations between somatic symptoms and quality of life. Methods: A website provided the general Dutch population access to a 30-day (3 times a day) diary study assessing 43 items related to health and well-being, which gave participants personalized feedback. Associations between somatic symptoms and quality of life were examined with a linear mixed model. Results: A total of 629 participants completed 28,430 assessments, with an average of 45 (sd=32) assessments per participant. Most participants (n=517, 82%) were women and 531 (84%) had high education. Almost 40% of the participants (n=247) completed enough assessments (t=68) to generate personalized feedback including temporal dynamics between well-being, health behavior, and emotions. Substantial between-person variability was found in the within-person association between somatic symptoms and quality of life. Conclusions: We successfully built an application for automated diary assessments and personalized feedback. The application was used by a sample of mainly highly educated women, which suggests that the potential of our intensive diary assessment method for large-scale health promotion is limited. However, a rich dataset was collected that allows for group-level and idiographic analyses that can shed light on etiological processes and may contribute to the development of empirical-based health promotion solutions.
Article
Full-text available
Background: Many studies on resilience have shown that people can succeed in preserving mental health after a traumatic event. Less is known about whether and how people can preserve subjective wellbeing in the presence of psychopathology. We examined to what extent psychopathology can co-exist with acceptable levels of subjective wellbeing and which personal strengths and resources moderate the association between psychopathology and wellbeing. Methods: Questionnaire data on wellbeing (Manchester Short Assessment of Quality of Life/Happiness Index), psychological symptoms (Depression Anxiety Stress Scales), and personal strengths and resources (humor, Humor Style questionnaire; empathy, Empathy Quotient questionnaire; social company; religion; daytime activities, Living situation questionnaire) were collected in a population-based internet study (HowNutsAreTheDutch; N = 12,503). Data of the subset of participants who completed the above questionnaires (n = 2411) were used for the present study. Regression analyses were performed to predict wellbeing from symptoms, resources, and their interactions. Results: Satisfactory levels of wellbeing (happiness score 6 or higher) were found in a substantial proportion of the participants with psychological symptoms (58% and 30% of those with moderate and severe symptom levels, respectively). The association between symptoms and wellbeing was large and negative (-0.67, P < .001), but less so in persons with high levels of self-defeating humor and in those with a partner and/or pet. Several of the personal strengths and resources had a positive main effect on wellbeing, especially self-enhancing humor, having a partner, and daytime activities. Conclusions: Cultivating personal strengths and resources, like humor, social/animal company, and daily occupations, may help people preserve acceptable levels of wellbeing despite the presence of symptoms of depression, anxiety, and stress.
Article
Full-text available
We still lack operative and theoretically founded definitions of what a personality disorder (PD) is, as well as empirically validated and feasible instruments to measure the disorder construct. The Temperament and Character Inventory (TCI) is the only personality instrument that explicitly distinguishes personality style and disordered functioning. Here, we seek to (1) confirm in a clinical sample that the character dimensions of the TCI capture a general construct of PD across all specific PD subtypes, (2) determine whether such core features can be used to detect the presence of PD, and (3) analyze whether such detection is affected by the presence and severity of Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) Axis I symptoms. Two hundred five anxious/depressed outpatients were evaluated with the Structural Clinical Interview for DSM-IV Axis I and II Disorders. Assessment also included the TCI, the Hamilton rating scales for depression and anxiety, and the Panic and Agoraphobia Scale. Sixty-one patients (29.8%) were diagnosed as having a DSM-IV PD. Self-directedness and Cooperativeness, but no other TCI dimensions, predicted the presence of PD (Nagelkerke R(2) = 0.35-0.45) and had a moderate diagnostic utility (kappa = 0.47-0.58) when Axis I symptoms were absent or mild. However, accuracy decreased in anxious or depressed patients. Our study supports the hypothesis of a disorder construct that is not related to the intensity of any specific PD subtype but which is common to all PDs. This construct relies largely on internal representations of the self revealing ineffectiveness and uncooperativeness.
Article
Full-text available
One of the ideological foundations of the modern welfare states is the belief that people can be made happier by providing them with better living conditions. This belief is challenged by the theory that happiness is a fixed 'trait', rather than a variable 'state'. This theory figures both at the individual level and at the societal level. The individual level variant depicts happiness as an aspect of personal character; rooted in inborn temperament or acquired disposition. The societal variant sees happiness as a matter of national character; embedded in shared values and beliefs. Both variants imply that a better society makes no happier people. Happiness can be regarded as a trait if it meets three criteria: (1) temporal stability, (2) cross-situational consistency, and (3) inner causation. This paper checks whether that is, indeed, the case. The theory that happiness is a personal-character-trait is tested in a (meta) analysis of longitudinal studies. The results are: (1) Happiness is quite stable on the short term, but not in the long run, neither relatively nor absoloutely. (2) Happiness is not insensitive to fortune or adversity. (3) Happiness is not entirely built-in: its genetic basis is at best modest and psychological factors explain only part of its variance. The theory that happiness is a national-character-trait is tested in an analysis of differences in average happiness between nations. The results point in the same direction: (1) Though generally fairly stable over the last decades, nation-happiness has changed profoundly in some cases, both absolutely and relatively. (2) Average happiness in nations is clearly not independant of living conditions. The better the conditions in a country, the happier its citizens. (3) The differences cannot be explained by a collective outlook on life. It is concluded that happiness is no immutable trait. There is thus still sense in striving for greater happiness for a greater number.
Book
As cognitive models of behavior continue to evolve, the mechanics of cognitive exceptionality, with its range of individual variations in abilities and performance, remains a challenge to psychology. Reaching beyond the standard view of exceptional cognition equaling superior intelligence, the Handbook of Individual Differences in Cognition examines the latest findings from psychobiology, cognitive psychology, and neuroscience, for a comprehensive state-of-the-art volume. Breaking down cognition in terms of attentional mechanisms, working memory, and higher-order processing, contributors discuss general models of cognition and personality. Chapter authors build on this foundation as they revisit current theory in such areas as processing effort and general arousal and examine emerging methods in individual differences research, including new data on the role of brain plasticity in cognitive function. The possibility of a unified theory of individual differences in cognitive ability and the extent to which these variables may account for real-world competencies are emphasized, and commentary chapters offer suggestions for further research priorities. Researchers, clinicians, and graduate students in psychology and cognitive sciences, including clinical psychology and neuropsychology, personality and social psychology, neuroscience, and education, will find the Handbook of Individual Differences in Cognition an expert guide to the field as it currently stands and to its agenda for the future.
Article
Empathy is an essential part of normal social functioning, yet there are precious few instruments for measuring individual differences in this domain. In this article we review psychological theories of empathy and its measurement. Previous instruments that purport to measure this have not always focused purely on empathy. We report a new self-report questionnaire, the Empathy Quotient (EQ), for use with adults of normal intelligence. It contains 40 empathy items and 20 filler/control items. On each empathy item a person can score 2, 1, or 0, so the EQ has a maximum score of 80 and a minimum of zero. In Study 1 we employed the EQ with n = 90 adults (65 males, 25 females) with Asperger Syndrome (AS) or high-functioning autism (HFA), who are reported clinically to have difficulties in empathy. The adults with AS/HFA scored significantly lower on the EQ than n = 90 (65 males, 25 females) age-matched controls. Of the adults with AS/HFA, 81% scored equal to or fewer than 30 points out of 80, compared with only 12% of controls. In Study 2 we carried out a study of n = 197 adults from a general population, to test for previously reported sex differences (female superiority) in empathy. This confirmed that women scored significantly higher than men. The EQ reveals both a sex difference in empathy in the general population and an empathy deficit in AS/HFA.
Article
The meaning of health is complex and subject to change. In this article,four conceptual models of health are presented to summarize the current meanings for health. The medical model is the most widely used definition in the United States, but the World Health Organization model has gained in popularity during the past several decades. In addition, there are other newer models-the wellness model and the environmental model-that are adding new meanings to the definition of health. By understanding and combining these different meanings, the prospects for improving medical outcomes and the quality of care are enhanced. This conceptual work is a prelude to improving health status assessment in a variety of contexts.
Book
Positive psychology exploded into public consciousness ten years ago and has continued to capture attention around the world ever since. The movement promised to study positive human nature, using only the most rigorous scientific tools and theories. How well has this promise been fulfilled? This book evaluates the first decade of this fledgling field of study from the perspective of nearly every leading researcher in the field. Scholars in the areas of social, personality, clinical, biological, emotional, and applied psychology take stock of their fields, while bearing in mind the original manifesto and goals of the positive psychology movement. Chapters provide honest, critical evaluations of the flaws and untapped potential of these various fields of study. The chapters design the optimal future of positive psychology by addressing gaps, biases, and methodological limitations, and exploring exciting new questions.