Content uploaded by Michelle Yik
Author content
All content in this area was uploaded by Michelle Yik on Aug 29, 2022
Content may be subject to copyright.
Content uploaded by Shalom H Schwartz
Author content
All content in this area was uploaded by Shalom H Schwartz on Apr 25, 2022
Content may be subject to copyright.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 1
Draft date: 31 December 2021
Yik, M., Mues, C., Sze, I. N. L., Kuppens, P., Tuerlinckx, F., De Roover, K., Kwok, F. H. C., Schwartz, S.
H., Abu-Hilal, M., Adebayo, D. F., Aguilar, P., Al-Bahrani, M., Anderson, M. H., Andrade, L., Bratko, D.,
Bushina, E., Choi, J. W., Cieciuch, J., Dru, V., . . . Russell, J. A. (2022). On the relationship between
valence and arousal in samples across the globe. Emotion. Advance online publication.
https://doi.org/10.1037/emo0001095
On the Relationship between Valence and Arousal in Samples across the Globe
Michelle Yik1
Chiel Mues2
Irene N. L. Sze1
Peter Kuppens2
Francis Tuerlinckx2
Kim De Roover3
Felity H. C. Kwok1
Shalom H. Schwartz4
Maher Abu-Hilal5, Damilola Fisayo Adebayo6, Pilar Aguilar7, Muna Al-Bahrani5, Marc H. Anderson8,
Laura Andrade9, Denis Bratko10, Ekaterina Bushina11, Jeong Won Choi12, Jan Cieciuch13 14, Vincent
Dru15, Uwana Evers16, Ronald Fischer17, Ivonne Andrea Florez18, Ragna B. Garðarsdóttir19, Aikaterini
Gari20, Sylvie Graf21, Peter Halama22, Jamin Halberstadt23, Magdalena S. Halim24, Renata M.
Heilman25, Martina Hřebíčková26, Johannes Alfons Karl17, Goran Knežević27, Michal Kohút28, Martin
Kolnes29, Ljiljana B. Lazarević27, Nadezhda Lebedeva11, Julie Lee16, Young-Ho Lee12, Chunquan Liu30,
Rasmus Mannerström31, Iris Marušić32, Florence Nansubuga33, Oluyinka Ojedokun6, Joonha Park34,
Tracey Platt35, René T. Proyer36, Anu Realo29 37, Jean-Pierre Rolland15, Willibald Ruch38, Desiree Ruiz7,
Florencia M. Sortheix39, Alexander Stahlmann38, Ana Stojanov23, Włodzimierz Strus13, Maya Tamir40,
Cláudio Torres9, Angela Trujillo41, Thi Khanh Ha Truong42, Akira Utsugi34, Michele Vecchione43, Lei
Wang30, and James A. Russell44
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 2
1 Division of Social Science, Hong Kong University of Science and Technology
2 Faculty of Psychology and Educational Sciences, KU Leuven-University of Leuven
3 Department of Methodology and Statistics, Tilburg School of Social and Behavioral Sciences
4 Department of Psychology, The Hebrew University of Jerusalem
5 College of Education, Sultan Qaboos University
6 Department of Psychology, Adekunle Ajasin University
7 Department of Psychology, Universidad Loyola Andalucia
8 Department of Management and Entrepreneurship, Iowa State University
9 Department of Basic Psychological Processes, University of Brasilia
10 Department of Psychology, University of Zagreb
11 School of Psychology, National Research University Higher School of Economics
12 Department of Psychology, The Catholic University of Korea
13 Institute of Psychology, Cardinal Wyszyński University in Warsaw
14 University Research Priority Program "Social Networks", University of Zurich
15 Laboratoire Interactions Cognition Action Émotion (LICAÉ), University Paris Nanterre
16 Department of Marketing, University of Western Australia
17 School of Psychology, Victoria University of Wellington
18 Behavioral Health Department, Kaiser Permanente
19 Faculty of Psychology, University of Iceland
20 Department of Psychology, National and Kapodistrian University of Athens
21 Department of Psychology, University of Bern
22 Centre of Social and Psychological Sciences, Slovak Academy of Sciences
23 Department of Psychology, University of Otago
24 Graduate Program of Professional Psychology, ATMA JAYA Catholic University of Indonesia
25 Department of Psychology, Babeș-Bolyai University
26 Institute of Psychology, Masaryk University
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 3
27 Department of Psychology, University of Belgrade
28 Department of Psychology, University of Trnava
29 Institute of Psychology, University of Tartu
30 School of Psychological and Cognitive Sciences, Peking University
31 Department of CICERO Learning, University of Jyväskylä
32 Centre for Educational Research and Development, Institute for Social Research in Zagreb
33 School of Psychology, Makerere University
34 Graduate School of Humanities, Nagoya University
35 Faculty of Health Sciences & Wellbeing, University of Sunderland
36 Department of Psychology, Martin-Luther-University Halle-Wittenberg
37 Department of Psychology, University of Warwick
38 Department of Psychology, University of Zurich
39 Swedish School of Social Sciences, University of Helsinki
40 Department of Psychology, The Hebrew University
41 Faculty of Psychology, Universidad de La Sabana
42 Faculty of Psychology, USSH, Vietnam National University
43 Department of Social and Developmental Psychology, Sapienza University of Rome
44 Department of Psychology and Neuroscience, Boston College
Author Note
Preparation of the paper was facilitated by the Hong Kong Research Grants Council’s General
Research Fund (Project Nos. 16651916 and 16601818).
Correspondence concerning this article should be addressed to Michelle Yik, Hong Kong
University of Science and Technology, Division of Social Science, Clear Water Bay, Kowloon, Hong
Kong. E-mail: Michelle.Yik@ust.hk
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 4
Abstract
Affect is involved in many psychological phenomena, but a descriptive structure, long sought, has
been elusive. Valence and arousal are fundamental, and a key question–the focus of the present
study–is the relationship between them. Valence is sometimes thought to be independent of
arousal, but, in some studies (representing too few societies in the world) arousal was found to vary
with valence. One common finding is that arousal is lowest at neutral valence and increases with
both positive and negative valence: a symmetric V-shaped relationship. In the study reported here of
self-reported affect during a remembered moment (N = 8,590), we tested the valence-arousal
relationship in 33 societies with 25 different languages. The two most common hypotheses in the
literature–independence and a symmetric V-shaped relationship–were not supported. With data of
all samples pooled, arousal increased with positive but not negative valence. Valence accounted for
between 5% (Finland) and 43% (China Beijing) of the variance in arousal. Although there is evidence
for a structural relationship between the two, there is also a large amount of variability in this
relation.
Keywords: valence, arousal, subjective experience, structure of affect, culture
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 5
On the Relationship between Valence and Arousal in Samples across the Globe
A person is “never in a state entirely free from feeling”.
Wundt (1897/1998, p. 92)
Affective feelings infuse mental processes and behaviors related to health, well-being,
psychopathology, and decision making. Yet, psychology has not achieved an agreed upon descriptive
structure of affect. Valence and arousal have often been identified as fundamental properties of
affect, but the relationship between the two has not been agreed upon or examined across the
globe. The present study focuses on momentary affect. We asked, in 33 different samples, two basic
questions: How are valence and arousal related to each other in subjective experience? Does this
relationship vary across societies? The study examined these questions by asking participants to
report their feelings in “a clearest moment” during the previous day.
Valence and Arousal
Valence (also known as pleasure-displeasure or hedonic tone) is an elementary dimension of
conscious affective feeling (Reisenzein, 1992; Wundt 1897/1998) and the most commonly found
fundamental property of affect (Larsen & Diener, 1992; Yik et al., 2002; Yik et al., 1999; Yik et al.,
2011) – indeed, sometimes the only factor found in self-reports of affect (Williams et al., 1989). Still,
controversy remains as to whether valence is one bipolar dimension or two separate dimensions (for
progress on this issue, see Larsen et al., 2001; Russell, 2017; Russell & Carroll, 1999; Yik, 2007).
Arousal (also known as activation, energy, or tension) often emerges as a second factor in self-
reported affect (Yik et al., 2002) and was prominent in earlier psychological writings (e.g., Berlyne,
1960; Cannon, 1927; Schachter & Singer, 1962). Self-reported arousal is related to a range of factors
from food to personality to neurochemistry (Thayer, 1989).
Theoretical Relations between Valence and Arousal
How valence and arousal are related to each other has received less attention, but is
essential, nonetheless. There are hints that arousal increases with intensity of both positive and
negative valence in certain conditions. For instance, arousal is a V-shaped function of valence in
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 6
studies of visual scenes (Lang, 1994; Mattek et al., 2017), in some emotion lexicons (Ćoso et al.,
2019; Yao et al., 2017), and in sentiment analysis of social media data (Chen & Yik, 2021). Perhaps
the V-shape occurs generally in all subjective experience. Alternatively, the valence-arousal
relationship might vary with domain, or with culture and language, or with individuals. When
attempting to map valence to certain brain regions such as orbitofrontal cortex and arousal to other
regions such as the amygdala, researchers have reported inconsistent findings across studies
(Colibazzi et al., 2010; Lindquist et al., 2012; Posner et al., 2009). Any variability in the valence-
arousal relationship might explain this inconsistency in studies of the neural basis of affect. In short,
the valence-arousal relationship in self-reported subjective experience needs to be better
understood.
Several relationships between valence and arousal in self-reported affect have been
suggested and tested (Kuppens et al., 2013; Kuppens et al., 2017). Prominent theoretical models are
displayed in Figure 1.
Model 1: Independence
Valence is often assumed to be independent of arousal in self-reported affect (e.g., Barrett &
Russell, 1999; Carver & Scheier, 1990; Larsen & Diener, 1992; Yik et al., 2011). In this model, how
pleasant or unpleasant one is feeling provides no information about how aroused one is feeling and
vice versa.
Model 2: Linear Relation
A second model posits a linear relationship, i.e., in the extreme, valence equals arousal. On
one version of this model–positive correlation version–affect is one dimension ranging from sadness
(negative valence, low arousal) to excitement (positive valence, high arousal). An interesting
possibility is that this model applies mainly to Western societies as reflected in a preference for
highly aroused pleasant affect (Tsai et al., 2006).
The alternative version of this model–negative correlation version–is that affect is one
dimension ranging from tension (negative valence, high arousal) to calmness (positive valence, low
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 7
arousal). This model was assumed in the psychoanalytic theory in which pleasure was thought to
originate from the release of tension and in the behaviorist theory that reinforcement is the
reduction of drive. An interesting possibility is that this model applies mainly to Asian societies as
reflected in a preference for deactivated pleasant affect (Tsai et al., 2006).
Model 3: Symmetric V-shaped Relation
In this model, arousal is minimal at neutral valence and then increases with (or is) the
intensity of positive and, separately, of negative valence. The relation is symmetric with positive and
negative valence having an equal intercept on the arousal axis and slope values equal in magnitude
but opposite in sign. Model 3 is commonly thought of as the V-shaped relationship shown in Figure
1. Here, we also allow Model 3 to include an inverted V-shaped relationship. Model 3 resonates with
Gray’s (1987) theory of two independent motivational systems–behavioral activation and inhibition–
in which arousal is the intensity of each system and with Thayer’s (1989) theory of two different
types of arousal, one positive and one negative.
Models 4 - 6: Asymmetric V-shaped Relation
Models 4 through 6 are based on the evaluative space model (Cacioppo & Berntson, 1994).
These models are similar to Model 3, but with asymmetries. Model 4 adds a positivity offset: positive
valence begins at a higher level of arousal than does negative valence. That is, the curves for positive
and negative valence have different intercepts on the arousal axis.
Model 5 adds a different asymmetry: differences in the slopes for positive versus negative
valence. For instance, Ito and Cacioppo (2005) argued that arousal increases more strongly with
negative valence than it does with positive valence (something that they called negativity bias). The
opposite can in principle also occur, namely that arousal increases more strongly with positive
valence. Both intercept and slope asymmetries appear in Model 6.
Cultural Variations in the Valence-Arousal Relationship
In addition to the different theoretical models, the empirical evidence in favor of or against
these models has been inconsistent as well (see Kuppens et al., 2013 for a detailed discussion). Part
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 8
of this inconsistency may arise because the relation between valence and arousal may differ with
the stimulus condition, with the culture or language, or even with the person examined. Indeed,
Kuppens et al. (2013) tested the six theoretical models by deploying multilevel regression models
that incorporate both a nomothetic (i.e., population) structure and idiographic variations in the
nomothetic structure (i.e., individual differences modeled as random effects). They found support
for asymmetric V-shaped relationships (Models 5 and 6) in eight samples of English speakers at the
nomothetic level, but the relationship at the population level was weak and showed large variations
at the idiographic level, implying perhaps the valence-arousal relationship can vary from one sample
to the next.
To complement the data from English-speaking societies, Kuppens et al. (2017) examined
data from another five societies. In contrast to prior findings, Kuppens et al. supported a symmetric
V-shaped relationship (Model 3) in all but Hong Kong (Model 1). The slope was steepest for Western
cultures (Canada, Spain) but less steep (Japan, Korea) to almost flat (Hong Kong) for Eastern cultures.
Clearly more cross-cultural data are needed. Therefore, in the present study, we sought to
test the six models on a large cross-cultural network involving 33 samples. They span six continents
and cover the global regions identified by Schwartz (2006).
Measurements of Self-reported Momentary Affect
The variety of measures used in the past studies complicated the examination of the
valence-arousal relationship. Valence and arousal have been measured in various ways such as with
the Self-Assessment Manikin (Bradley & Lang, 2007) or the Affect Grid (Russell et al., 1989). Kuppens
et al. (2013) used items tapping pleasure, displeasure, high arousal, and low arousal. In the present
study, we adopted Kuppens et al.’s method by asking the participants to report their affect using
affect items covering pleasant, unpleasant, activated, and deactivated. We then tested structural
invariance of the two constructs, namely valence (defined by pleasant and unpleasant items) and
arousal (defined by activated and deactivated items), across the 33 samples.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 9
The instructions for self-reported affect in the past studies have been problematic.
Sometimes, the participant was asked about his or her affect over an extended period of time
(today, this week, etc.), but affective feelings ebb and flow, sometimes changing quickly. Participants
were sometimes asked to rate their feelings during a specific type of remembered episode or their
reactions to a set of stimuli such as tunes or pictures; such ratings have restricted variance and likely
lack the social complexity of everyday life. In other cases, participants simply responded to a
questionnaire, with questions such as “How are you feeling right now?”. The variance in such ratings
is likely restricted because all participants are in the same circumstance, such as filling out a
questionnaire, or perhaps sitting in a boring lab.
Here we focus on momentary affect. To capture everyday momentary feelings, experience
sampling would be ideal, although it can be costly thereby becoming a stumbling block to large-scale
cross-cultural projects. An alternative to experience sampling is to measure affect in a broader range
of moments. In the present study, a “remembered moments” questionnaire (RMQ) was used in
which participants recalled a clear moment from the day before (see Yik et al., 2002; see the Day
Reconstruction Method developed by Kahneman et al. [2004]). The moments from the RMQ method
are likely to be varied and representative of experiences outside the lab. Of course, memory is
fallible, and so the RMQ is designed to have the participant select a well-remembered moment.
Method
Samples and Participants
The 33 datasets collected cover six continents, using translations into 25 different languages
including Indo-European (Croatian, Czech, Dutch, English, French, German, Greek, Icelandic, Italian,
Polish, Portuguese, Romanian, Russian, Serbian, Slovak, and Spanish), Afro-Asiatic (Arabic, Hebrew),
Uralic (Estonian, Finnish), Austroasiatic (Vietnamese), Austronesian (Indonesian), Japonic (Japanese),
Koreanic (Korean), and Sino-Tibetan (Chinese). For feasibility, we intended to recruit 200 participants
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 10
per sample.
1
A total of 8,590 university students (59% female) took part in the study during February
to November 2018. Sample sizes ranged from 190 (Belgium, Nigeria) to 469 (Czech Republic). All
participants were at least 16 years of age, with an overall mean of 24.01 years (SD = 7.67). For the
demographic characteristics of the samples, please refer to the online supplemental materials 1.
Procedure
We are a team of researchers involved in cross-cultural projects (see McCrae et al., 2005). All
researchers involved in this project are fluent in English and have extensive experience collaborating
in large-scale survey research projects and translating questionnaires into their own languages. For
non-English speaking samples, each researcher received an English questionnaire package for
translation purpose. A standardized translation and back-translation procedure was used to prepare
different language versions. For each language, we recruited two bilinguals; the first bilingual
translated the English items into the target language and the second bilingual independently back-
translated the items into English. Discrepancies between the original and back-translated English
versions were identified, discussed, and reconciled.
Participants were asked to complete nine questionnaires including the one reported in this
article; average completion time was 35 minutes. Most data were collected online using Qualtrics
(25 samples), with a few samples using the paper-pencil method (4 samples), or both methods (4
samples). The study was approved by the HKUST Human Participants Research Panel. All data were
collected in accordance with the local ethical guidelines and procedures.
2
Upon the completion of data collection, collaborators provided details on the sample
description, the data collection method, and unexpected events, if any, during the data collection.
1
After data collection, we found that a power analysis using an effect size of 8% found in Study 1c in
Kuppens et al. (2013) indicated that the sample of 200 achieves .95 power with α = .05.
2
University ethics approval was required and obtained in 11 samples.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 11
The initial sample consisted of 8,642 participants among whom 52 cases were excluded in data
screening resulting in a final sample of 8,590 participants for subsequent analysis.
3
Instructions and Measures
Participants were asked to recall a clearly remembered moment from the day before:
“Please think back to yesterday. Search your memory for a particular moment that is especially clear
in your memory. Let’s call it your clearest moment.” To help the participants to relive the moment,
they were asked to think about the time, location, the person they were with, and things that they
were doing during this clearest moment.
They then rated their feelings during that moment using 16 affect adjectives. The 16
adjectives were culled from four affect segments of the 12-Point Affect Circumplex (12-PAC; Yik et
al., 2011).
4
Valence was tapped by four pleasant items (“happy”, “pleased”, “content”, “satisfied”)
and four unpleasant items (“miserable”, “unhappy”, “troubled”, “dissatisfied”), whereas arousal was
tapped by five activated items (“determined”, “intense”, “hyperactivated”, “aroused”, “activated”)
and three deactivated items (“still”, “quiet”, “sleepy”). Participants rated their affect on a 5-point
Likert scale ranging from 1 (not at all) to 5 (extremely). The median values of the alpha coefficients
ranged from .50 (deactivated) to .93 (pleasant). For details, please refer to the online supplemental
materials 2.
Results and Discussion
The data were processed and analyzed in four steps: In a first step, we determined
measurement-invariant scales for valence and arousal across the 33 samples. In a second step, we
calculated valence and arousal scores per participant. The resulting data were then analyzed to
determine the valence-arousal relation and cultural variations therein. To this end, in a third step,
3
Four were eliminated because they left blank all items on at least four questionnaires
administered. Another 48 were eliminated because they used the same response option for all items
for at least two of the nine questionnaires.
4
Male and female versions were developed for the affect measure in 14 of 25 languages
where there are masculine and feminine adjectives.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 12
we first fit models each representing a different theoretical model to the data of each sample
separately, and then identified the dominant patterns across the 33 samples. In a final and fourth
step, we fit a (multilevel) model that allows for these dominant patterns across all participants from
all samples (N = 8,590). This model allows us to identify what an overall, pancultural relation
between valence and arousal would look like, how much of the total variance such a pancultural
model could explain the degree to which each sample might deviate from this overall model.
Step 1: Measurement Invariance
To evaluate the invariance of measures for valence and arousal across the 33 samples, we
tested configural invariance (factor loadings and intercepts freely estimated across groups) and
metric invariance (factor loadings constrained to be equal across groups) to ensure the meaning of
the latent construct was equal across groups. For details of the procedure, please refer to the online
supplemental materials 3.
Figure 2 presents the final model consisting of 11 items with two correlated residuals. The
model fit of the metric invariance model was compared with the configural model. Metric invariance
across the samples is indicated when imposing invariant factor loadings leads to no more than .02
decrease in CFI, no more than .03 increase in RMSEA (Rutkowski & Svetina, 2014), and no more
than .02 increase in SRMR (Chen, 2007). The changes of the fit measures between the two models
were small (ΔCFI = .018, ΔRMSEA = .003, ΔSRMR = .033) indicating that the factor loadings are equal
across groups and thus metric invariance holds. Therefore, the comparison of the linear valence-
arousal relation across the samples can be carried out. Significant positive covariances were
observed in all 33 samples, with covariances ranging from .19 (Finland) to .86 (Indonesia) in the
metric invariance model. For details, please refer to the online supplemental materials 4 and 5. This
step resulted in the identification of the items to define a valence score and an arousal score per
participant.
Step 2: Calculation of the Final Valence and Arousal Scores
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 13
The theoretical and mathematical models used to capture the various possible relations
between valence and arousal make use of a neutral valence midpoint (forming the deflection point
of any asymmetric relation). Consequently, it was not possible to use the factor scores from the
abovementioned final factor analytic model as input for the analyses modeling the relation between
valence and arousal, as the point at which the factor scores equal zero cannot be assumed to reflect
neutral valence. To circumvent this problem, using the 11 items in Figure 2, we calculated valence
and arousal scores per participant by subtracting the average of the negative valence items from the
average of the positive valence items, and the low arousal item from the average of the high arousal
items respectively (similar approach was used by Kuppens et al., 2013, 2017).
Step 3: Best Fitting Model for Each Sample
We first examined the relationship between these valence and arousal scores within each
sample. In each sample, we fit six different statistical regression models in which arousal was
modelled as a function of valence in correspondence with the theoretical relations from Figure 1.
The models we fit to the data, however, allowed more variation of values than those shown in Figure
1. For example, the “Model 3” we fit to the data allowed an inverted V as well as the V-shape shown
in Figure 1; the “Model 5” we fit to the data allowed various slope values as well as the steeper slope
for negative valence shown in Figure 1. In addition, for more flexibility, we included an additional
nonparametric model (Model 7) that does not make prior parametric assumptions (see Kuppens et
al., 2013 for more details on the statistical models).
To select the model that provides the most appropriate fit to the data in each sample, we
relied on the Bayesian Information Criterion (BIC) and posterior model probabilities derived from the
BIC (Raftery, 1995). The best fitting model has the lowest BIC score and highest posterior model
probability (see Kuppens et al., 2013). For each sample, the seven models were estimated,
separately, and the best fitting model was selected. Table 1 presents the model selection indices and
Table 2 the best fitting model for each sample. Figure 3 shows the plotted data between valence and
arousal together with the best fitting model separately for each sample.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 14
No one model showed the best fit in all 33 samples: different samples were best
characterized by different models. However, only four models emerged as best fitting. Model 2
emerged in 14 samples, with arousal increasing linearly with valence. In the remaining 19 samples,
models including an asymmetry were selected: 16 samples included a V-shape relationship with a
steeper slope for positive valence (Model 5), two included a higher intercept for positive valence
(Model 4), and one included both a lower intercept and a steeper slope for positive valence (Model
6).
In short, two models dominated in 30 of the 33 samples: Model 2 in 14 samples and Model 5
in 16 samples. In about half of the samples, slopes differed by valence: Positive valence uniformly
showed a strong positive slope with arousal, but negative valence showed slopes ranging from
negative to flat to positive.
The finding of the main support for Models 2 and 5 should be understood against the
background of the large variations in the valence-arousal relationship within each sample (as evident
in the scattered data points in the 33 plots in Figure 3). As shown in the next-to-last column of Table
2, the variance accounted for by the best fitting model was often low, with R2 values ranging
from .05 (Model 2 for Finland) to .43 (Model 5 for China Beijing). Thus, explanatory power of even
the best fitting model in which arousal is a function of valence was often low, and within each
sample there remains much variation around the overall relation.
Step 4: One Model for all 33 Samples
We next evaluated the possibility of one pancultural model to describe the relation between
valence and arousal. As a first step in exploring this possibility, we collapsed the data across samples.
With the pooled data of 8,590, we fit the seven theoretical models to the data. The results are
shown in the first line of Table 1. Our version of Model 5 (in which we allowed empirically
determined values for the two slopes) provided the best fit. Another consideration also favored
Model 5: Within each separate sample, both Model 5 and Model 2 had emerged as best fitting.
Nonetheless, Model 2 can be thought of as a special case of Model 5: Model 2 adds the constraint
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 15
that the slope of negative valence is equal in magnitude to the slope of positive valence. Model 5 is
thus the more general model, and the frequency of finding an asymmetry in slope values led us to
Model 5.
5
Thus, Model 5 is the best candidate for a nomothetic structure of affect. The version of
Model 5 that fit the total sample is shown in the thick black line of Figure 4. As can be seen in this
figure, this version features a positive slope for positive valence, and an almost flat slope for
negative valence.
To account for between-sample differences in this overall relation, a multilevel extension of
Model 5 was also estimated. The multilevel framework allowed us to model an overall, population-
average relation between valence and arousal across the data from all 33 samples (i.e., the fixed
effects structure), and at the same time to estimate sample-specific deviations from this average
relation (i.e., the random effects structure). Indeed, the fixed effects pertaining to the intercept and
slope values of this population average-model reveal the shape of the average model across all
samples, and the random effects pertaining to the intercepts and slopes allow for variation between
samples. Table 3 shows the numerical estimates of the model, and Figure 5 displays the estimated
fixed effects part of the model, portraying the population average model across all 33 samples,
together with sample-specific deviations. Across all data, the marginal R2 of this model (i.e., the
proportion of variance explained by the fixed effects alone) equals .19, and the conditional R2 (i.e.,
the proportion of variance explained by both the fixed and random factors) equals .25 (see
Nakagawa et al., 2017).
6
These results mean that taking the fixed part of the model only (i.e.,
assuming equal intercept and slopes across samples) explains 19% of the total variance observed
5
Model 6 was also a reasonable candidate to explore as a pan-cultural model for all
samples. After all, Model 6 is the most general of the models. Model 6 is equivalent to Model 5 but
allows different intercepts for positive and negative valence (viz., an offset). The offset occurred in
the best fitting model for only three of the 33 samples, and in one of those the value of the intercept
was opposite to that predicted by Cacioppo and Berntson (1994). Offset thus seemed to be an
unlikely feature of the general model we sought.
6
We opted here for a naive calculation of R2 in the multilevel model by calculating r2
(observed y, predicted y), which is the squared correlation between the observations and the
predictions from the model. The predictions come from the fixed effects part of the model only (i.e.,
the marginal R2) or from the fixed plus random effects part (i.e., the conditional R2).
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 16
across all participants. Allowing sample-specific deviations in these parameters increases this value
to 25% of the total variance.
The multilevel extension of Model 5 underscores three points about the population-
average, pancultural model. First, as shown in Table 3 and in the thick black line in Figure 5, the
intercept of the model is close to the arousal midpoint (i.e., not significantly different from zero).
Second, the model contains different slopes for positive and negative valence. Specifically, the slope
of negative valence is flat whereas the slope of positive valence was significantly steeper than the
slope for negative valence. Three, despite this overall relation, there is variation among the samples
in the parameters (see also the thin lines in Figure 5). The appearance of variation was confirmed by
the sizeable sample-specific deviations from the fixed effects structure as indicated by the variance
components of the multilevel model. There is considerable variation across samples for the
intercept, the negative valence slope, and the positive valence slope.
Finally, one may wonder how allowing for sample-specific deviations (as in the sample
specific models or the multilevel model reported above) compares to an approach that would
assume the exact same relation between valence and arousal in every sample. To evaluate this
possibility, we estimated the proportion of explained variance per sample if one would fit the same
model dictated by the fixed effects portion of Model 5 (with a positive slope for positive valence and
a flat slope for negative valence; see Table 3 and the bold line in Figures 4 and 5) to the data from
each sample. To do so, we examined the squared correlation between the observed arousal values
and the arousal values predicted by the fixed effects component of the multilevel model. The R2
values are reported in the last column of Table 2. The possibility of a single model with no sample-
specific parameters was supported by the similarity of these R2 values to those from the separate
different models per country (the last second column of Table 2).
Conclusion
Model 5–our version as seen in the thick black line of Figure 4 or 5–provides a reasonably
good fitting general model, an average global relationship between valence and arousal across our
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 17
33 samples and among our 8,590 participants. Such a general model is useful for many purposes. If,
of a group of participants, all we know is that they are human, then this version of Model 5 is a good
basis for the description of their momentary affect: When they are feeling pleasant, they tend to be
feeling activated; When they are feeling unpleasant, they could be feeling activated, deactivated, or
in between. More generally, the average global relationship between valence and arousal is
asymmetric, with an almost flat slope for negative valence that is joined to a positive slope for
positive valence. Model 5 is the prime candidate for a universal pancultural account of the general
relation between valence and arousal.
The flat slope for negative valence might possibly be explained, in part, by two reasons. One
was related to memory recall. Our participants tended to remember positive valence, and this effect
is vividly evident in all 33 plots in Figure 3.
7
In the most extreme case, people in Oman almost never
recalled any negative valence. Certainly this does not mean that they never experience anything
negative. Rather they did not report negative valence. Positivity bias in memory recall was well
documented in the literature supporting the prevalence of pleasant (vs. unpleasant) events (see
Botzung et al., 2010). Others have found that the affect associated with unpleasant memories fades
faster than that associated with pleasant memories (see Ritchie et al., 2015). The intersection
between memory recall and the valence-arousal model should be included in future research
directions.
The second reason for the flat slope could be related to the word choice. In English, most of
the words used to anchor arousal appear to be positive (e.g., “determined”, and “aroused”). The
positivity of these words could bias our results to show that arousal and positive valence are
correlated, but arousal and negative valence are not. When we chose the words to define arousal in
English, we sought to focus on those saturated with arousal and relatively independent of valence.
(To maintain the independence of valence and arousal in the translations, detailed instructions were
7
1,589 subjects fell in the negative valence region where the correlation between valence and arousal
was .02 (p = .40).
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 18
given to the translators in the remaining 24 languages.) To test the positivity bias of the arousal
words, we estimated the correlations between the four high arousal items (“determined”,
“aroused”, “hyperactivated”, “activated”) and the positive valence score in each of the 33 samples:
The mean correlation was .32 (SD = .15) for “determined”; .35 (SD = .20) for “aroused”, .41 (SD = .09)
for “activated”, and .30 (SD = .14) for “hyperactivated”. (In the US sample, the corresponding values
were .06, .32, .27, and .13.) These positive correlations might be due to the co-occurrence of higher
arousal with positive valence, or to a semantic relationship such that these four high arousal words
have some component of positive valence, or to the memory bias discussed above. Our results lent
some potential support that word choice is one possible explanation for our global model, but co-
occurrence of positive valence and arousal is also possible. The differences in correlations across the
four arousal items (.30 to .41) are consistent with both factors influencing the results. So, for now,
we can conclude that positive valence is positively correlated with arousal. It remains for future
research to determine how much of that correlation is due to general co-occurrence of positive
valence and arousal in daily life, how much to semantics of the items used, and how much to a
memory bias as discussed above.
A more precise model is possible for each separate sample. That is, as also shown in Figure
5, we also found evidence for differences among the 33 samples – differences that can be
represented simply by three parameters: (1) the value of arousal at neutral valence, (2) the slope of
arousal as a function of positive valence, and (3) the slope of arousal as a function of negative
valence. Specifying values for each of these three parameters for a specific sample provides a better
fitting model of affect. Why each parameter takes on the value it does for a given sample remains to
be seen, however. This question rises to the top of the list of important directions for future
research. Of course, simply sampling differences might have occurred, but more interesting
possibilities are differences between samples in terms of personality, culture, language, geography,
and social differences.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 19
One possibility–suggested by results shown in Figure 3–is that more than these three
parameters are needed because the relationship of valence to arousal varies even more with sample
when modeled separately. As shown in Figure 3, for example, the best fitting model for both Nigeria
and Russia has a positivity offset and a positive slope of arousal as a function of the intensity of
negative valence – a combination of features not seen in the best fitting model for any other society.
The best fitting model for 19 samples has an inflection point such that the arousal slope changes
from negative to positive valence, and yet no inflection point occurred for the other 14 samples.
Such differences are more likely than the three parameters of Figure 5 to be due to sampling
differences, and yet they are hints of interesting possibilities. In these cases, replicability is the first
question.
Certain negative conclusions are also warranted. In no sample did the independence model
(Model 1 of Figure 1) provide the best fit. This finding in itself is important, as it indicates that the
model most commonly presupposed in measures of self-reported affect is only an approximation.
On the other hand, for no sample did valence account for a large amount of variance in arousal
scores. In other words, the consistently low values of R2 for even the best fitting model in each
sample and the overall model in all 8,590 participants support a fair amount of independence
between valence and arousal. In addition, for the single best average model of Figure 4 or 5, arousal
was independent of negative valence. That average model or the version with three parameters
must therefore be interpreted against the background of the low degree of predictive strengths of
the best fitting model within each sample. Therefore, these prevalent relationships seen here should
be interpreted in a probabilistic rather than deterministic manner. For any individual, any
combination between valence and arousal remains possible.
Further, we found no evidence indicating a need for highly complex models to represent the
relation between valence and arousal. Of the 7 models examined, only 4 emerged as the best fit for
even one sample. Even more telling, in no sample did the non-parametric Model 7 provide the best
fit. The relation between valence and arousal within and between samples can be represented by
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 20
simple principles. We offer the simple model seen in Figure 4 or 5, with three parameters to
represent sample differences, as the most promising account consistent with current evidence.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 21
References
Barrett, L. F., & Russell, J. A. (1999). The structure of current affect: Controversies and emerging
consensus. Current Directions in Psychological Science, 8(1), 10-14.
https://doi.org/10.1111/1467-8721.00003
Berlyne, D. E. (1960). Conflict, arousal, and curiosity. McGraw-Hill Book Company.
https://doi.org/10.1037/11164-000
Botzung, A., Rubin, D. C., Miles, A., Cabeza, R., & LaBar, K. S. (2010). Mental hoop diaries: Emotional
memories of a college basketball game in rival fans. Journal of Neuroscience, 30(6), 2130-
2137.
Bradley, M. M., & Lang, P. J. (2007). Emotion and motivation. In J. T. Cacioppo, L. G. Tassinary, & G.
G. Berntson (Eds.), Handbook of psychophysiology (pp. 581-607). Cambridge University
Press. https://doi.org/10.1017/CBO9780511546396.025
Cacioppo, J. T., & Berntson, G. G. (1994). Relationship between attitudes and evaluative space: A
critical review, with emphasis on the separability of positive and negative substrates.
Psychological Bulletin, 115(3), 401-423. https://doi.org/10.1037/0033-2909.115.3.401
Cannon, W. B. (1927). The James-Lange theory of emotions: A critical examination and an alternative
theory. The American Journal of Psychology, 39(1/4), 106-124.
https://doi.org/10.2307/1415404
Carver, C. S., & Scheier, M. F. (1990). Origins and functions of positive and negative affect: A control-
process view. Psychological Review, 97(1), 19-35. https://doi.org/10.1037/0033-
295X.97.1.19
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance.
Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464-504.
https://doi.org/10.1080/10705510701301834
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 22
Chen, X., & Yik, M. (2021). The valence and arousal trajectories throughout the Wuhan lockdown
during COVID-19: Evidence from sentiment analysis using Weibo data [Manuscript in
preparation]. Division of Social Science, Hong Kong University of Science and Technology.
Colibazzi, T., Posner, J., Wang, Z., Gorman, D., Gerber, A., Yu, S., Zhu, H., Kangarlu, A., Duan, Y.,
Russell, J. A., & Peterson, B. S. (2010). Neural systems subserving valence and arousal during
the experience of induced emotions. Emotion, 10(3), 377-389.
https://doi.org/10.1037/a0018484
Ćoso, B., Guasch, M., Ferré, P., & Hinojosa, J. A. (2019). Affective and concreteness norms for 3,022
Croatian words. Quarterly Journal of Experimental Psychology, 72(9), 2302-2312.
https://doi.org/10.1177/1747021819834226
Gray, J. A. (1987). The psychology of fear and stress (2nd ed.). Cambridge University Press.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:
Conventional criteria versus new alternatives. Structural Equation Modeling: A
Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:
Conventional criteria versus new alternatives. Structural Equation Modeling: A
Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
Ito, T., & Cacioppo, J. T. (2005). Variations on a human universal: Individual differences in positivity
offset and negativity bias. Cognition & Emotion, 19(1), 1-26.
https://doi.org/10.1080/02699930441000120
Kahneman, D., Krueger, A. B., Schkade, D. A., Schwarz, N., & Stone, A. A. (2004). A survey method for
characterizing daily life experience: The day reconstruction method. Science, 306(5702),
1776-1780. https://doi.org/10.1126/science.1103572
Kuppens, P., Tuerlinckx, F., Russell, J. A., & Barrett, L. F. (2013). The relationship between valence
and arousal in subjective experience. Psychological Bulletin, 139(4), 917-940.
https://doi.org/10.1037/a0030811
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 23
Kuppens, P., Tuerlinckx, F., Yik, M., Koval, P., Coosemans, J., Zeng, K. J., & Russell, J. A. (2017). The relation
between valence and arousal in subjective experience varies with personality and culture. Journal
of Personality, 85(4), 530-542. https://doi.org/10.1111/jopy.12258
Lang, P. J. (1994). The motivational organization of emotion: Affect-reflex connections. In S. H. M. van
Goozen, N. E. Van de Poll, & J. A. Sergeant (Eds.), Emotions: Essays on emotion theory (pp. 61-93).
Psychology Press. https://doi.org/10.4324/9781315806914
Larsen, R. J., & Diener, E. (1992). Promises and problems with the circumplex model of emotion. In M. S.
Clark (Ed.), Review of personality and social psychology, No. 13. Emotion (pp. 25-59). Sage
Publications, Inc.
Larsen, J. T., McGraw, A. P., & Cacioppo, J. T. (2001). Can people feel happy and sad at the same
time? Journal of Personality and Social Psychology, 81(4), 684-696.
http://dx.doi.org/10.1037/0022-3514.81.4.684
Lindquist, K. A., Wager, T. D., Kober, H., Bliss-Moreau, E., & Barrett, L. F. (2012). The brain basis of
emotion: A meta-analytic review. Behavioral and Brain Sciences, 35(3), 121-143.
https://doi.org/10.1017/S0140525X11000446
Mattek, A. M., Wolford, G. L., & Whalen, P. J. (2017). A mathematical model captures the structure of
subjective affect. Perspectives on Psychological Science, 12(3), 508-526.
https://doi.org/10.1177/1745691616685863
McCrae, R. R., Terracciano, A., Khoury, B., Nansubuga, F., Knežević, G., Jocic, D. D., Ahn, H., Ahn, C., De
Fruyt, F., Gülgöz, S., Ruch, W., Ghayur, M. A., Avia, M. D., Sánchez-Bernardos, M. L., Rossier, J.,
Dahourou, D., Fischer, R., Shakespeare-Finch, J., Yik, M., . . . Camart, N. (2005). Universal features
of personality traits from the observer’s perspective: Data from 50 cultures. Journal of Personality
and Social Psychology, 88(3), 547-561. https://doi.org/10.1037/0022-3514.88.3.547
Owe, E., Vignoles, V. L., Becker, M., Brown, R., Smith, P. B., Lee, S. W. S., Easterbrook, M., Gadre, T.,
Zhang, X., Gheorghiu, M., Baguma, P., Tatarko, A., Aldhafri, S., Zinkeng, M. Schwartz, S. J., Des
Rosiers, S. E., Villamar, J. A., Mekonnen, K. H., Regalia, C., ... Jalal, B. (2013). Contextualism as an
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 24
important facet of individualism–collectivism: Personhood beliefs across 37 national groups.
Journal of Cross–Cultural Psychology, 44(1), 24-45. https://doi.org/10.1177/0022022111430255
Nakagawa, S., Johnson, P. C. D., & Schielzeth, H. (2017). The coefficient of determination R2 and intra-
class correlation coefficient from generalized linear mixed-effects models revisited and expanded.
Journal of the Royal Society Interface, 14(134), 1-11. https://doi.org/10.1098/rsif.2017.0213
Pinheiro, J. C., & Bates, D. M. (2000). Mixed-effects models in S and S-PLUS. Springer.
https://doi.org/10.1007/b98882
Posner, J., Russell, J. A., Gerber, A., Gorman, D., Colibazzi, T., Yu, S., Wang, Z., Kangarlu, A., Zhu, H., &
Peterson, B. S. (2009). The neurophysiological bases of emotion: An fMRI study of the
affective circumplex using emotion‐denoting words. Human Brain Mapping, 30(3), 883-895.
https://doi.org/10.1002/hbm.20553
R Development Core Team. (2005). R: A language and environment for statistical computing. R
Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org
Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111-
163. https://doi.org/10.2307/271063
Reisenzein, R. (1992). A structuralist reconstruction of Wundt's three-dimensional theory of
emotion. In H. Westmeyer (Ed.), The structuralist program in psychology: Foundations and
applications (pp. 141-189). Hogrefe & Huber Publishers.
Ritchie, T. D., Batteson, T. J., Bohn, A., Crawford, M. T., Ferguson, G. V., Schrauf, R. W., Vogl, R. J., &
Walker, W. R. (2015). A pancultural perspective on the fading affect bias in autobiographical
memory. Memory, 23(2), 278-290.
Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling and more. Version 0.5-12
(BETA). Journal of Statistical Software, 48(2), 1-36.
Russell, J. A. (2017). Mixed emotions viewed from the psychological constructionist perspective.
Emotion Review, 9(2), 111-117. https://doi.org/10.1177/1754073916639658
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 25
Russell, J. A., & Carroll, J. M. (1999). On the bipolarity of positive and negative affect. Psychological
Bulletin, 125(1), 3-30. https://doi.org/10.1037/0033-2909.125.1.3
Russell, J. A., Weiss, A., & Mendelsohn, G. A. (1989). Affect Grid: A single-item scale of pleasure and
arousal. Journal of Personality and Social Psychology, 57(3), 493-502.
https://doi.org/10.1037/0022-3514.57.3.493
Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the
context of large-scale international surveys. Educational and Psychological Measurement,
74(1), 31-57. https://doi.org/10.1177/0013164413498257
Schachter, S., & Singer, J. (1962). Cognitive, social, and physiological determinants of emotional
state. Psychological Review, 69(5), 379-399. https://doi.org/10.1037/h0046234
Schwartz, S. H. (2006). A theory of cultural value orientations: Explication and applications.
Comparative Sociology, 5(2-3), 137-182. https://doi.org/10.1163/156913306778667357
Thayer, R. E. (1989). The biopsychology of mood and arousal. Oxford University Press.
Tsai, J. L., Knutson, B., & Fung, H. H. (2006). Cultural variation in affect valuation. Journal of
Personality and Social Psychology, 90(2), 288-307. https://doi.org/10.1037/0022-
3514.90.2.288
Van der Leeden, R., Meijer, E., & Busing F. M. T. A. (2008) Resampling Multilevel Models. In J. de
Leeuw & E. Meijer (Eds), Handbook of multilevel analysis (pp. 401-433). Springer.
https://doi.org/10.1007/978-0-387-73186-5_11
Williams, L. J., Cote, J. A., & Buckley, M. R. (1989). Lack of method variance in self-reported affect
and perceptions at work: Reality or artifact? Journal of Applied Psychology, 74(3), 462-468.
https://doi.org/10.1037/0021-9010.74.3.462
Wundt, W. (1998). Outlines of psychology (C. H. Judd, Trans.). Thoemmes Press. (Original work
published 1897). https://doi.org/10.1037/12908-000
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 26
Yao, Z., Wu, J., Zhang, Y., & Wang, Z. (2017). Norms of valence, arousal, concreteness, familiarity,
imageability, and context availability for 1,100 Chinese words. Behavior Research Methods,
49(4), 1374-1385. https://doi.org/10.3758/s13428-016-0793-2
Yik, M. (2007). Culture, gender, and the bipolarity of momentary affect. Cognition and Emotion,
21(3), 664-680. https://doi.org/10.1080/02699930600823702
Yik, M., Russell, J. A., Ahn, C., Fernandez‐Dols, J. M., & Suzuki, N. (2002). Relating the five‐factor
model of personality to a circumplex model of affect: A five language study. In R. R. McCrae
& J. Allik (Eds.), The Five‐Factor Model of personality across cultures (pp. 79‐104). Springer.
https://doi.org/10.1007/978-1-4615-0763-5_5
Yik, M., Russell, J. A., & Barrett, L. F. (1999). Structure of self-reported current affect: Integration and
beyond. Journal of Personality and Social Psychology, 77(3), 600-619.
https://doi.org/10.1037/0022-3514.77.3.600
Yik, M., Russell, J. A., & Steiger, J. H. (2011). A 12–point circumplex structure of core affect. Emotion,
11(4), 705-731. https://doi.org/10.1037/a0023980
VALANCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 27
Table 1
Summary of Model Selection Indices when Arousal is Modeled as a Function of Valence
Model 1
Model 2
Model 3
Model 4
Model 5
Model 6
Model 7
Independence
Linear Relation
Symmetric V
Asymmetric V
with different
intercepts
Asymmetric V
with different
slopes
Asymmetric V
with different
intercepts,
different slopes
Nonparametric
Region/Samplea
BIC
PostP
BIC
PostP
BIC
PostP
BIC
PostP
BIC
PostP
BIC
PostP
BIC
PostP
Full Data Set
33532.27
.00
31805.94
.00
32422.25
.00
31843.74
.00
31554.87
.99
31563.24
.01
31586.78
.00
Africa and the Middle East
Nigeria
620.66
.00
572.08
.01
569.05
.05
563.71
.79
568.34
.08
568.77
.06
573.76
.01
Oman
815.38
.00
774.73
.70
777.57
.17
782.70
.01
778.43
.11
783.70
.01
784.42
.01
Uganda
699.26
.00
601.22
.69
633.63
.00
607.67
.03
603.69
.20
607.62
.03
606.51
.05
Confucian
China (Beijing)
822.59
.00
717.84
.01
770.85
.00
727.10
.00
709.39
.89
714.74
.06
715.54
.04
China (Hong Kong)
1035.41
.00
960.35
.00
986.77
.00
955.55
.01
946.10
.88
950.68
.09
954.02
.02
Japan
1013.45
.00
989.64
.75
1001.35
.00
997.95
.01
992.33
.20
997.85
.01
996.71
.02
South Korea
1056.80
.00
1028.53
.00
1015.52
.11
1019.60
.01
1012.20
.58
1014.01
.24
1016.77
.06
East Europe
Croatia
899.84
.00
860.07
.17
869.86
.00
859.48
.23
858.07
.46
863.19
.04
861.14
.10
Czech Republic
1942.60
.00
1848.63
.09
1915.72
.00
1874.53
.00
1844.70
.62
1846.88
.21
1848.73
.08
Estonia
856.17
.00
823.89
.26
833.50
.00
824.47
.20
822.73
.47
827.49
.04
828.67
.02
Poland
1560.32
.00
1507.40
.42
1545.42
.00
1508.94
.19
1507.97
.31
1511.87
.04
1512.82
.03
Romania
931.11
.00
856.26
.74
929.75
.00
891.14
.00
860.75
.08
859.13
.18
864.75
.01
Russia
987.44
.00
961.12
.05
969.14
.00
956.93
.43
956.93
.43
961.13
.05
962.65
.02
Serbia
883.41
.00
815.00
.00
834.88
.00
809.00
.03
802.19
.90
807.23
.07
819.26
.00
Slovakia
1007.14
.00
966.60
.03
980.91
.00
965.23
.07
960.18
.82
965.65
.05
966.57
.03
VALANCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 28
English-Speaking
Australia
940.79
.00
906.05
.88
922.22
.00
920.66
.00
911.57
.06
917.02
.00
911.49
.06
Israel
833.46
.00
781.06
.30
799.38
.00
789.53
.00
779.68
.61
784.92
.04
785.02
.04
New Zealand
1671.69
.00
1631.36
.10
1645.39
.00
1640.11
.00
1627.38
.75
1631.84
.08
1632.16
.07
UK (England)
790.38
.00
748.26
.45
781.11
.00
759.26
.00
748.27
.45
753.51
.03
752.37
.06
United States
994.00
.00
968.94
.76
981.48
.00
972.87
.11
972.65
.12
976.76
.02
985.19
.00
Latin America
Brazil
918.63
.00
865.85
.72
902.75
.00
886.33
.00
868.28
.21
872.85
.02
871.15
.05
Colombia
1066.42
.00
937.70
.20
1018.39
.00
962.68
.00
935.13
.74
940.73
.04
943.28
.01
South Asia
Indonesia
1392.45
.00
1266.55
.12
1344.17
.00
1298.72
.00
1262.66
.84
1268.51
.04
1280.67
.00
Vietnam
925.21
.00
882.56
.80
902.38
.00
900.27
.00
887.39
.07
891.48
.01
886.38
.12
West Europe
Belgium
723.13
.00
700.96
.03
698.10
.12
697.02
.20
694.84
.60
700.05
.04
702.99
.01
Finland
840.68
.02
834.17
.63
836.12
.24
841.25
.02
838.41
.08
841.80
.01
843.66
.01
France
1093.98
.00
1063.91
.01
1069.29
.00
1060.92
.04
1054.67
.86
1059.75
.07
1061.59
.03
Germany
837.19
.00
803.55
.50
825.45
.00
804.26
.35
807.75
.06
808.91
.03
808.30
.05
Greece
1208.86
.00
1122.73
.00
1151.27
.00
1128.22
.00
1101.11
.43
1100.56
.57
1111.10
.00
Iceland
1228.70
.00
1204.55
.03
1200.01
.33
1203.51
.06
1199.24
.49
1203.68
.05
1204.39
.04
Italy
933.98
.00
866.30
.50
909.68
.00
886.42
.00
866.56
.44
871.18
.04
873.62
.01
Spain
807.61
.00
766.31
.03
769.99
.00
764.73
.06
759.36
.85
764.64
.06
774.81
.00
Switzerland
862.32
.00
827.23
.84
849.61
.00
837.45
.01
831.08
.12
836.52
.01
834.10
.03
Note. BIC = Bayesian Information Criterion (lower values reflect better fit). PostP indicates posterior probability of each model given the data among the set
of seven models. The fit indices of the best-fitting are underlined and bold.
a Global regions were identified by Schwartz (2006).
VALANCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 29
Table 2
Overview of Best Fitting Model for the Relation between Valence and Arousal (in Comparison with the Fixed Effects part of the Multilevel Extension of Model
5)
Region/Samplea
Best Model
Relation
Higher Intercept
Steeper Slope
R2 for the Best
Fitting Model
R2 based on the
Fixed Effects of
Model 5b
Africa and the Middle East
Nigeria
4
Asymmetric V
Positive valence
--
.21
.18
Oman
2
Linear / positive
--
--
.14
.14
Uganda
2
Linear / positive
--
--
.24
.25
Confucian
China (Beijing)
5
Asymmetric V
--
Positive valence
.43
.43
China (Hong Kong)
5
Asymmetric V
--
Positive valence
.31
.31
Japan
2
Linear / positive
--
--
.11
.12
South Korea
5
Asymmetric V
--
Positive valence
.19
.17
East Europe
Croatia
5
Asymmetric V
--
Positive valence
.18
.18
Czech Republic
5
Asymmetric V
--
Positive valence
.21
.21
Estonia
5
Asymmetric V
--
Positive valence
.18
.18
Poland
2
Linear / positive
--
--
.13
.14
Romania
2
Linear / positive
--
--
.30
.29
Russia
4
Asymmetric V
Positive valence
--
.16
.16
Serbia
5
Asymmetric V
--
Positive valence
.33
.33
Slovakia
5
Asymmetric V
--
Positive valence
.21
.21
VALANCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 30
English-Speaking
Australia
2
Linear / positive
--
--
.15
.14
Israel
5
Asymmetric V
--
Positive valence
.27
.27
New Zealand
5
Asymmetric V
--
Positive valence
.12
.12
UK (England)
2
Linear / positive
--
--
.21
.23
United States
2
Linear / positive
--
--
.11
.12
Latin America
Brazil
2
Linear / positive
--
--
.22
.23
Colombia
5
Asymmetric V
--
Positive valence
.41
.41
South Asia
Indonesia
5
Asymmetric V
--
Positive valence
.33
.32
Vietnam
2
Linear / positive
--
--
.13
.12
West Europe
Belgium
5
Asymmetric V
--
Positive valence
.18
.18
Finland
2
Linear / positive
--
--
.05
.05
France
5
Asymmetric V
--
Positive valence
.16
.15
Germany
2
Linear / positive
--
--
.16
.16
Greece
6
Asymmetric V
Negative valence
Positive valence
.33
.31
Iceland
5
Asymmetric V
--
Positive valence
.12
.12
Italy
2
Linear / positive
--
--
.27
.28
Spain
5
Asymmetric V
--
Positive valence
.22
.21
Switzerland
2
Linear / positive
--
--
.16
.16
a Global regions were identified by Schwartz (2006).
b See Table 3 for the multilevel extension of Model 5.
VALANCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 31
Table 3
Results of the Multilevel Extension of Model 5
Fixed Effects
Random Effects (Variance components)
Estimate
SE
t (df = 8,446)
p
SD
95% CI
Intercept
–.10
.07
–1.49
.137
.32
[.21, .43]
Negative valence slope
–.03
.03
–1.08
.279
.08
[.03, .14]
Positive valence slope
.55
.03
18.94
< .001
.13
[.07, .18]
Note. The 95% CI for the SD of the random effects are generated using a parametric bootstrap procedure (Van der Leeden et al., 2008, p. 410; see also
Pinheiro & Bates, 2000).
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 32
Figure 1
Six Possible Relations Between Valence and Arousal
Model 1 Model 2 Model 3
Model 4 Model 5 Model 6
Note. Model 1 is the independence model: Valence is independent of arousal. In Model 2, a linear
relation is assumed allowing for valence to increase linearly with arousal. Model 3 assumes a
symmetric V-shaped relation so that arousal may increase with the intensity of positive, and
separately, of negative valence. Model 4 permits an asymmetric relation with different intercepts so
that positive valence may begin at a higher level of arousal than does negative valence. In Model 5,
an asymmetric V-shaped relation with different slopes is assumed; arousal may increase more
strongly with negative compared with positive valence or vice versa. Model 6 combines Models 4
and 5 resulting in an asymmetric V-shaped relation with different intercepts and different slopes.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 33
Figure 2
The Final Two-Factor Model for which Metric Invariance Holds
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 34
Figure 3
Relationship Between Valence and Arousal in Each Sample with the Best Fitting Model
Note. The panels are ordered from the simplest to the most complex models. Colors are used to
differentiate between the best models (red for Model 2 for Oman to Vietnam, green for Model 4 for
Nigeria and Russia, blue for Model 5 for China Beijing to Spain, and purple for Model 6 for Greece).
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 35
Figure 4
Relationship between Valence and Arousal based on an Overall, non-multilevel Model 5
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 36
Figure 5
Relationship between Valence and Arousal based on the Multilevel Extension of Model 5
Note. The population average (i.e., fixed effects) is shown as the thick black line; the sample-specific
relations (fixed plus random effects) are shown as the colored thin lines.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 37
Supplemental Materials 1
Demographic Characteristics of the 33 Samples
Age
Sample
Language
Schwartz’s (2006) Region
Data Collection
N
% Female
% Native
Mdn
M
(SD)
Australia
English
English-Speaking
Qualtrics
251
60%
96%
29.00
29.86
(8.47)
Belgium
Dutch
West Europe
Qualtrics
190
88%
97%
18.00
19.17
(3.18)
Brazil
Brazilian Portuguese
Latin America
Qualtrics & Paper
220
58%
100%
21.00
22.52
(5.20)
China (Beijing)
Simplified Chinese
Confucian
Qualtrics
220
60%
100%
21.00
21.29
(2.83)
China (Hong Kong)
Traditional Chinese
Confucian
Qualtrics
272
43%
98%
21.00
21.20
(1.03)
Colombia
Colombia Spanish
Latin America
Qualtrics
270
54%
99%
21.00
21.22
(2.42)
Croatia
Croatian
East Europe
Paper
227
54%
98%
21.00
20.76
(1.52)
Czech Republic
Czech
East Europe
Qualtrics
469
71%
87%
23.00
24.74
(6.65)
Estonia
Estonian
East Europe
Qualtrics
227
60%
94%
23.00
26.03
(7.73)
Finland
Finnish
West Europe
Qualtrics
240
63%
97%
26.00
27.35
(6.89)
France
French
West Europe
Paper
272
62%
99%
21.00
21.64
(2.90)
Germany
Germany German
West Europe
Paper
232
70%
96%
22.00
24.03
(7.45)
Greece
Greek
West Europe
Qualtrics
317
56%
97%
19.00
19.74
(3.66)
Iceland
Icelandic
West Europe
Qualtrics
316
73%
96%
28.00
32.73
(12.71)
Indonesia
Indonesian
South Asia
Qualtrics & Paper
349
47%
97%
21.00
21.06
(2.56)
Israel
Hebrew
English-Speaking
Qualtrics
209
59%
99%
25.00
25.09
(4.02)
Italy
Italian
West Europe
Qualtrics
237
61%
97%
22.00
23.64
(5.13)
Japan
Japanese
Confucian
Qualtrics
251
42%
98%
19.00
19.12
(1.15)
New Zealand
English
English-Speaking
Qualtrics
426
57%
98%
19.00
19.74
(3.61)
Nigeria
English
Africa and the Middle East
Paper
190
47%
98%
21.00
21.69
(3.40)
Oman
Arabic
Africa and the Middle East
Qualtrics
245
60%
94%
30.00
32.17
(11.69)
Poland
Polish
East Europe
Qualtrics
404
70%
94%
33.00
33.81
(10.99)
Romania
Romanian
East Europe
Qualtrics
226
57%
95%
21.00
23.71
(7.07)
Russia
Russian
East Europe
Qualtrics
240
63%
98%
20.00
21.67
(5.99)
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 38
Serbia
Serbian
East Europe
Qualtrics
228
45%
97%
21.00
22.42
(3.19)
Slovakia
Slovak
East Europe
Qualtrics
246
57%
98%
22.00
23.55
(4.67)
South Korea
Korean
Confucian
Qualtrics
269
65%
98%
20.00
21.14
(3.08)
Spain
Spain Spanish
West Europe
Qualtrics & Paper
202
59%
100%
21.00
21.66
(4.22)
Switzerland
Swiss German
West Europe
Qualtrics
238
65%
93%
28.00
31.05
(10.04)
Uganda
English
Africa and the Middle East
Qualtrics & Paper
206
52%
90%
21.00
21.83
(3.92)
UK (England)
English
English-Speaking
Qualtrics
199
54%
94%
28.50
31.85
(12.33)
United States
English
English-Speaking
Qualtrics
264
53%
95%
20.00
20.74
(1.91)
Vietnam
Vietnamese
South Asia
Paper
238
55%
100%
20.00
20.48
(1.57)
Full Data Set
8,590
59%
96%
21.00
24.01
(7.67)
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 39
Supplemental Materials 2
Reliabilities of the Four Affect Segments Defining Valence and Arousal
Sample
N
Pleasant
Unpleasant
Activated
Deactivated
Australia
251
.83
.89
.77
.59
Belgium
190
.92
.91
.74
.01
Brazil
220
.94
.88
.82
.46
China (Beijing)
220
.94
.93
.76
.57
China (Hong Kong)
272
.95
.91
.75
.61
Colombia
270
.90
.91
.84
.50
Croatia
227
.94
.88
.77
.46
Czech Republic
469
.97
.91
.83
.66
Estonia
227
.96
.85
.75
.65
Finland
240
.94
.92
.75
.60
France
272
.95
.88
.81
.24
Germany
232
.94
.91
.64
.42
Greece
317
.95
.94
.75
.41
Iceland
316
.94
.92
.81
.45
Indonesia
349
.88
.76
.66
.62
Israel
209
.93
.89
.80
.67
Italy
237
.92
.89
.87
.43
Japan
251
.88
.88
.76
.50
New Zealand
426
.92
.92
.77
.59
Nigeria
190
.75
.76
.47
.39
Oman
245
.82
.71
.49
.35
Poland
404
.94
.95
.77
.13
Romania
226
.96
.91
.79
.64
Russia
240
.95
.86
.75
.41
Serbia
228
.93
.89
.81
.33
Slovakia
246
.95
.92
.82
.66
South Korea
269
.93
.91
.83
.69
Spain
202
.87
.84
.75
.64
Switzerland
238
.94
.88
.65
.34
Uganda
206
.85
.89
.53
.32
UK (England)
199
.93
.93
.76
.55
United States
264
.92
.90
.72
.62
Vietnam
238
.88
.87
.78
.51
Full Data Set
8,590
Median α
.93
.89
.76
.50
Weighted average α
.92
.89
.75
.49
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 40
Supplemental Materials 3
Testing Measurement Invariance of Valence and Arousal
To define valence, we began with the affect items capturing pleasant, and unpleasant
segments of the 12-Point Affect Circumplex (12-PAC; Yik et al., 2011); to define arousal, we began
with the affect items capturing activated, and deactivated segments. The configural invariance
model (factor loadings and intercepts freely estimated across groups) tested whether the pattern of
zero and non-zero loadings for the factors was equal across groups whereas the metric invariance
model (factor loadings constrained to be equal across groups) tested whether the meaning of the
latent constructs was equal across groups. When the assumptions of metric invariance were
satisfied, the valence-arousal relations could be compared across the 33 samples.
To evaluate the goodness of fit for the invariance models, the comparative fit index (CFI),
root mean square error of approximation (RMSEA), and standardized root-mean-square residual
(SRMR) statistics were examined. CFI values of at least .95, RMSEA values smaller than or equal
to .06, and SRMR values small than or equal to .08 are typically considered good fit (Hu & Bentler,
1999), although Rutkowski and Svetina (2014) showed that a cut-off of .10 is more appropriate for
the RMSEA in case of 10 or more groups. When moving from a less restricted (i.e., configural) to a
more restricted (i.e., metric) model, Chen (2007) postulated that a difference of less than .01 in CFI
(ΔCFI), .02 in RMSEA (ΔRMSEA), or .03 in SRMR (ΔSRMR) indicates invariance. However, Rutkowski
and Svetina showed that more liberal criteria should be used when multiple groups are involved, i.e.,
ΔCFI less than .02 and ΔRMSEA less than .03 for establishing metric invariance.
To assess measurement invariance of valence and arousal, we conducted a series of
multigroup confirmatory factor analyses (MGCFA) using the lavaan package (version 0.6-7.1564,
Rosseel, 2012) for R software (R Development Core Team, 2005). Missing data were estimated using
Full Information Maximum Likelihood estimation. To test the configural model, a two-factor model
was examined in which valence was tapped by four pleasant items (“happy”, “pleased”, “content”,
“satisfied”) and four unpleasant items (“miserable”, “unhappy”, “troubled”, “dissatisfied”) whereas
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 41
arousal by five activated items (“determined”, “intense”, “hyperactivated”, “aroused”, “activated”)
and three deactivated items (“still”, “quiet”, “sleepy”). This hypothesized model revealed a poor fit
in the configural invariance model: CFI = .77, RMSEA = .15, SRMR = .13.
Following Owe et al. (2013), we refined the scales by (1) eliminating items with non-
significant loadings, and (2) examining the largest modification indices and correlating pairs of
residuals wherever appropriate. To improve the model fit, items with low loadings would be further
examined and excluded to simplify the model. After incorporating each of these modifications,
model fit was re-examined until an acceptable model fit was reached.
Five items were excluded from the model. The item “still” had non-significant loadings in 11
samples, followed by “intense” (5 samples) and “quiet” (5 samples). These three items were
removed together with items “miserable” and “troubled” which had the lowest loadings on the
valence factor in 27 and 26 samples, respectively. In addition, the modification indices revealed that
large error covariances were found between two pairs of items (“unhappy” and “dissatisfied”,
“content” and “satisfied”) and that the correlations between their residuals were estimated in the
final models.
The final model consisted of 11 items with two correlated residuals, with a substantial
improvement on model fit on the configural invariance model: CFI = .94, RMSEA = .10, SRMR = .08.
Each latent construct was defined by its items, with item loadings exceeding |.18| and differed
significantly from zero (ps < .05) in all samples except Australia, Japan, Nigeria, and Oman. Using the
final model, metric invariance was tested by restricting all item loadings to be equal across groups.
The model fit of the metric invariance was then compared with the configural model. The metric
invariance model revealed an acceptable fit: CFI = .92, RMSEA = .10, SRMR = .11. The changes of the
fit measures between the two models were small (ΔCFI = .018, ΔRMSEA = .003, ΔSRMR = .032)
indicating that the factor loadings are equal across groups (metric invariance) and, thus, justifying
the comparison of the valence-arousal relation across the 33 samples.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 42
Supplemental Materials 4
Standardized Maximum Likelihood Estimates for Factor Loadings of the Configural Model
Valence
Arousal
Sample
“happy”
“pleased”
“content”
“satisfied”
“unhappy”
“dissatisfied”
“determined”
“aroused”
“hyperactivated”
“activated”
“sleepy”
Australia
.76
.79
.62
.78
–.32
–.29
.56
.61
.78
.62
–.06
Belgium
.81
.86
.88
.90
–.68
–.71
.41
.74
.79
.78
–.32
Brazil
.88
.89
.93
.85
–.64
–.69
.63
.71
.77
.83
–.38
China (Beijing)
.92
.92
.87
.87
–.71
–.73
.54
.84
.56
.78
–.55
China (Hong Kong)
.92
.95
.86
.89
–.52
–.57
.41
.92
.56
.75
–.51
Colombia
.72
.79
.94
.90
–.68
–.75
.45
.79
.92
.72
–.35
Croatia
.92
.89
.90
.82
–.75
–.80
.60
.64
.85
.69
–.49
Czech Republic
.91
.93
.96
.93
–.81
–.82
.71
.71
.84
.84
–.47
Estonia
.91
.95
.89
.90
–.63
–.77
.61
.73
.80
.39
–.35
Finland
.86
.92
.92
.89
–.66
–.65
.31
.71
.92
.49
–.23
France
.89
.92
.94
.87
–.75
–.75
.67
.66
.65
.85
–.50
Germany
.81
.92
.93
.90
–.69
–.73
.56
.56
.90
.48
–.46
Greece
.87
.97
.84
.91
–.80
–.85
.47
.88
.90
.64
–.31
Iceland
.90
.94
.84
.89
–.78
–.82
.55
.55
.79
.77
–.46
Indonesia
.80
.88
.85
.70
–.38
–.47
.47
.51
.71
.82
–.31
Israel
.81
.80
.93
.93
–.65
–.68
.59
.67
.86
.68
–.38
Italy
.82
.82
.95
.90
–.71
–.80
.74
.56
.71
.92
–.35
Japan
.65
.81
.88
.90
–.56
–.60
.11
.85
.80
.70
–.19
New Zealand
.87
.90
.76
.88
–.72
–.69
.54
.71
.87
.48
–.30
Nigeria
.82
.77
.45
.68
–.71
–.56
.32
.30
.81
.40
–.16
Oman
.72
.80
.67
.71
–.38
–.36
.54
.48
.63
.07
–.45
Poland
.87
.93
.82
.89
–.77
–.81
.66
.40
.73
.61
–.46
Romania
.90
.93
.93
.95
–.80
–.85
.61
.79
.84
.63
–.52
Russia
.82
.95
.95
.92
–.73
–.72
.65
.78
.79
.73
–.46
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 43
Serbia
.84
.88
.93
.86
–.76
–.80
.67
.70
.89
.73
–.37
Slovakia
.89
.92
.87
.94
–.83
–.85
.69
.74
.87
.52
–.49
South Korea
.89
.95
.77
.88
–.58
–.55
.53
.76
.82
.72
–.23
Spain
.83
.61
.96
.85
–.67
–.70
.64
.58
.72
.71
–.46
Switzerland
.79
.91
.94
.94
–.73
–.84
.49
.49
.90
.56
–.47
Uganda
.89
.76
.74
.84
–.75
–.75
.49
.47
.81
.44
–.42
UK (England)
.83
.94
.84
.89
–.78
–.74
.56
.68
.84
.69
–.33
United States
.82
.91
.74
.93
–.79
–.72
.50
.59
.81
.39
–.34
Vietnam
.81
.87
.71
.83
–.42
–.45
.37
.69
.83
.63
–.18
Note. All ps < .05 except for “determined” (Japan), “aroused” (Oman), and “sleepy” (Australia, Nigeria).
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 44
Supplemental Materials 5
Standardized Parameter Estimates for Valence–Arousal Covariances
Valence–Arousal covariance
Sample
Configural
Metric
Australia
.52
.51
Belgium
.55
.54
Brazil
.67
.64
China (Beijing)
.70
.70
China (Hong Kong)
.61
.63
Colombia
.71
.74
Croatia
.56
.56
Czech Republic
.52
.51
Estonia
.51
.50
Finland
.25
.19
France
.55
.54
Germany
.44
.46
Greece
.59
.60
Iceland
.32
.30
Indonesia
.88
.86
Israel
.53
.55
Italy
.78
.72
Japan
.57
.51
New Zealand
.30
.32
Nigeria
.66
.70
Oman
.66
.55
Poland
.51
.40
Romania
.58
.58
Russia
.39
.38
Serbia
.52
.52
Slovakia
.43
.41
South Korea
.40
.41
Spain
.63
.60
Switzerland
.49
.50
Uganda
.66
.66
UK (England)
.56
.56
United States
.37
.39
Vietnam
.49
.49
Note. All ps < .05. A two–factor model was tested in each sample. Valence was defined by “happy”,
“pleased”, “content”, “satisfied”, “unhappy”, and “dissatisfied”; Arousal was defined by
“determined”, “aroused”, “hyperactivated”, “activated”, and “sleepy”. Two pairs of residual scores,
“unhappy” and “dissatisfied”, as well as “content” and “satisfied”, were correlated.
VALENCE AND AROUSAL IN SUBJECTIVE EXPERIENCE 45
References
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance.
Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464-504.
https://doi.org/10.1080/10705510701301834
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:
Conventional criteria versus new alternatives. Structural Equation Modeling: A
Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
Owe, E., Vignoles, V. L., Becker, M., Brown, R., Smith, P. B., Lee, S. W. S., Easterbrook, M., Gadre, T.,
Zhang, X., Gheorghiu, M., Baguma, P., Tatarko, A., Aldhafri, S., Zinkeng, M. Schwartz, S. J.,
Des Rosiers, S. E., Villamar, J. A., Mekonnen, K. H., Regalia, C., ... Jalal, B. (2013).
Contextualism as an important facet of individualism–collectivism: Personhood beliefs
across 37 national groups. Journal of Cross–Cultural Psychology, 44(1), 24-45.
https://doi.org/10.1177/0022022111430255
R Development Core Team. (2005). R: A language and environment for statistical computing. R
Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org
Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling and more. Version 0.5-12
(BETA). Journal of Statistical Software, 48(2), 1-36.
Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the
context of large–scale international surveys. Educational and Psychological Measurement,
74(1), 31-57. https://doi.org/10.1177/0013164413498257
Schwartz, S. H. (2006). A theory of cultural value orientations: Explication and applications.
Comparative Sociology, 5(2-3), 137-182. https://doi.org/10.1163/156913306778667357
Yik, M., Russell, J. A., & Steiger, J. H. (2011). A 12–point circumplex structure of core affect. Emotion,
11(4), 705-731. https://doi.org/10.1037/a0023980
A preview of this full-text is provided by American Psychological Association.
Content available from Emotion
This content is subject to copyright. Terms and conditions apply.