Conference PaperPDF Available

A Multimodal Workflow for Modeling Personality And Emotions to Enable User Profiling and Personalisation


Abstract and Figures

The Personality Emotion Model (PEM) is a workflow for generating quantifiable and bi-directional mappings between 15 personality traits and the basic emotions. PEM utilises Affective computing methodology to map this relationship across the modalities of self-report, facial expressions, semantic analysis, and affective prosody. The workflow is an end-to-end solution integrating data collection, feature extraction, data analysis, and result generation. PEM results in a real-time model that provides a high-resolution correlated mapping between personality traits and the basic emotions. The robustness of PEM's model is supported by the work-flow's ability to conduct meta-analytical and multimodal analysis; each state-to-trait mapping is dynamically updated in terms of its magnitude, direction, and statistical significance as data is processed. PEM provides a methodology that can contribute to long-standing research questions in the fields of Psychology and Affective computing. These research questions include (i) quantifying the emotive nature of personality, (ii) minimising the effects of context variance in basic emotions research, and (iii) investigating the role of emotion sequencing effects in relation to individual differences. PEM's methodology enables direct applications in any domain that requires the provision of individualised and personalised services (e.g. advertising, clinical care, research).
Content may be subject to copyright.
A Multimodal Workflow for Modeling Personality And Emotions to
Enable User Profiling and Personalisation
Ryan Donovan, Aoife Johnson , Aine deRoiste and Ruairi O’Reilly
Cork Institute of Technology, Cork, Ireland,,,
Keywords: Automatic Personality Recognition, Affective Computing, Personality, Five Factor Model, Emotion, Basic
Emotions, User Personalisaiton, User Profiling.
Abstract: The Personality Emotion Model (PEM) is a workflow for generating quantifiable and bi-directional mappings
between 15 personality traits and the basic emotions. PEM utilises Affective computing methodology to
map this relationship across the modalities of self-report, facial expressions, semantic analysis, and affective
prosody. The workflow is an end-to-end solution integrating data collection, feature extraction, data analysis,
and result generation. PEM results in a real-time model that provides a high-resolution correlated mapping
between personality traits and the basic emotions. The robustness of PEM’s model is supported by the work-
flow’s ability to conduct meta-analytical and multimodal analysis; each state-to-trait mapping is dynamically
updated in terms of its magnitude, direction, and statistical significance as data is processed. PEM provides a
methodology that can contribute to long-standing research questions in the fields of Psychology and Affective
computing. These research questions include (i) quantifying the emotive nature of personality, (ii) minimising
the effects of context variance in basic emotions research, and (iii) investigating the role of emotion sequencing
effects in relation to individual differences. PEM’s methodology enables direct applications in any domain that
requires the provision of individualised and personalised services (e.g. advertising, clinical care, research).
The interdisciplinary field of Personality science is
concerned with identifying, taxonomizing, and un-
derstanding the key aspects of personality. Person-
ality traits are the foundation for Personality science
(McAdams and Pals, 2006) (John et al., 2008). Traits
are the cognitive, behavioural, emotional, and motiva-
tional characteristics of a person that are stable across
situations (McAdams, 2015). Each person varies
along a continuum on any given trait (e.g. a person
can range from highly introverted, to a cross between
introverted and extraverted, to highly extraverted).
And while a person’s traits can and do change during
their lifetime, the person’s relative position on each
trait’s continuum remains stable (e.g. a highly ex-
traverted child will also be a highly extraverted adult,
even if their absolute levels of extraversion has de-
creased (Little, 2014)). Therefore, personality traits
provide information about the enduring and unique
structure of a person’s psyche relative to other peo-
1Personality is hereafter used as a shorthand for the col-
lection of traits a person has. For information about the
There is an increasing demand for individualised
and personalised technology and services that better
realise and meet the idiosyncratic needs of each in-
dividual (Vinciarelli and Mohammadi, 2014b) (Sub-
ramanian et al., 2016). Personality science is a valu-
able tool for meeting this demand, as personality traits
indicate the characteristic way a person thinks, acts,
feels, and desires relative to other people. Personality
enables researchers and professionals to apply client-
centred approaches given the personal characteristics
of their client-base, thereby gaining an advantage over
those that use blanketed approaches. (El Bachari
et al., 2010).
Personality is displayed via a variety of detectable
cues (Mehl et al., 2006). These cues span across mul-
tiple modalities, ranging from central-nervous sys-
tem activity, speech and language patterns, behaviour
across a variety of settings (e.g. consumer behaviour,
workplace behaviour), and self/peer report. To har-
ness the potential of personality in user adaptation,
there is a need to identify cues that (i) reliably indicate
personality across situations and (ii) can be detected
other aspects of personality, see (McCrae and Costa Jr,
automatically and accurately.
This paper argues that emotions meet both of these
conditions. The relationship between personality and
emotions has been (i) likened to that between the
climate and the weather: what one expects is per-
sonality [climate], what one observes at any par-
ticular moment is emotion [weather]” (Revelle and
Scherer, 2004). A significant body of both psycho-
logical and neuropsychological research supports this
analogy (Wilson et al., 2017) (Davis and Panksepp,
2018). Emotions can be (ii) detected automatically
across multiple modalities (e.g. writing, expressions,
neurophysiological activity, behaviour, etc). There-
fore emotions provide an opportunity for automatic
personality recognition and categorisation research.
The field of Affective computing (AC) is well-
positioned to avail of this opportunity. AC is con-
cerned with the development of technological systems
that can automatically detect, process, categorise, and
display human emotion (Picard, 2000). Significant
advances in automatic emotion detection have led to
successful AC applications in multiple domains, such
as in healthcare and education (Poria et al., 2017)
(D’Mello et al., 2018). The relationship between
personality and emotions provides an avenue for AC
research to extend its reach. AC methodology en-
ables automatic extraction of emotive-states, which
can be statistically correlated with personality traits,
to gain information about the enduring characteristics
of users.
The goal of the Personality Emotion Modelling
(PEM) workflow is to provide an end-to-end solu-
tion that enables researchers to map personality and
emotions via AC methodology at scale. This work-
flow identifies the key components for both person-
ality and emotion that have been validated and are
readily accessible via predefined interfaces. Addition-
ally, this workflow is automated in that the appropri-
ate AC methodology for extracting and categorising
emotive-states would be built into the workflow, en-
abling researchers to upload their data for analysis.
The analysed data would then be correlated with the
personality-traits of their participant sample and the
resulting state-trait mappings are visualised in terms
of their magnitude, significance, and direction.
The remainder of the article is organised as fol-
lows. Section 2 defines the key psychological phe-
nomena of the workflow. Section 3 presents the work-
flow of the model. Section 4 discusses the value of
PEM in contributing to open research questions and
the sustainability of the workflow. Section 5 discusses
the potential applications of PEM and provides a di-
rect use case scenario. Section 6 summarises the work
and presents conclusions.
Personality traits are typical expressions of emo-
tion, cognition, behaviour, and motivation across time
(Costa and McCrae, 1995). Personality traits reli-
ably and accurately predict important life outcomes
concerning health, relationships, and career success
(Soto, 2019). Personality traits represent our unique
general temperament or disposition towards the world
(DeYoung, 2015).
The Five Factor Model (FFM) is considered an
overarching paradigm from which to view Person-
ality science (John et al., 2008). The FFM pro-
vides a taxonomy of personality based on five broad
traits: Openness to Experience, Conscientiousness,
Extraversion, Agreeableness, and Neuroticism (Mc-
Crae and John, 1992). These five traits are consid-
ered to be at the highest level of the trait hierarchy
with each factor being an aggregate of multiple lower-
level facets (i.e. a specific aspect of personality) that
correlate highly with one another (e.g. Agreeable-
ness is made up of facets such as empathetic, co-
cooperativeness, and modesty (Graziano and Tobin,
2009)). This aggregation provides a greater band-
width that strengthens the accuracy of predictions
across situations, but it weakens the accuracy of pre-
dictions in specific situations (Soto and John, 2017).
A potential solution to this limitation is the provi-
sioning of an intermediate level between the FFM and
their facets. This intermediate level would (i) retain
adequate generalisability to assess cross-situational
psychological consistency (ii) but would also be “nar-
row” enough to enable specific behavioural predic-
tion. Candidates for such an intermediate level have
been identified; psychometric and genetic research
have demonstrated that each FFM trait can be broken
down into two further aspects, defined here as “sub-
traits” (DeYoung et al., 2007).
The benefit of using this intermediate level has
been demonstrated in experimental studies (Allen
et al., 2018) (Leon et al., 2017). For example, re-
searchers that previously investigated the relation-
ship between the FFM factor Agreeableness and po-
litical orientation found inconsistent results. How-
ever, research that incorporated the sub-trait hierarchy
demonstrated that the two sub-traits of Agreeableness
- Compassion and Politeness - correlated with left-
wing and right-wing political orientation respectively
(Hirsh et al., 2010). The importance of sub-traits in
identifying new links between personality and life-
outcomes has also been showcased in clinical and ed-
ucational domains (Allen et al., 2018) (Leon et al.,
2017). These are promising indications that sub-traits
Figure 1: The Five Factor Model’s five broad traits (Openness to Experience, Conscientiousness, Extraversion, Agreeableness,
and Neuroticism; OCEAN) and the associated sub-traits as specified by (DeYoung et al., 2007). The five broad traits represent
the highest level of the personality trait hierarchy. The foundational level of the trait hierarchy are facets, which aggregate
to form each of the five broad factors. The sub-traits operate in an intermediate level between these two levels and enable a
higher resolution analysis of an individuals personality.
are an integral component for assessing the influence
of personality in applied research settings.
Therefore, the PEM workflow incorporates the
sub-trait level of personality to (a) enable investiga-
tion of the particular influence of sub-traits in user-
tailoring and targeting and (b) to assess the claim
that the sub-traits offers a higher level of specificity
than the FFM without sacrificing generalisability. If
(a) and (b) can be empirically demonstrated, then
this supports the presupposition that this intermedi-
ate level is the next necessary component for future
FFM theories. PEM’s primary focus is to assess both
(a) and (b) in relation to the basic emotions.
2.1 The Basic Emotions
Basic Emotion Theory (BET (Hutto et al., 2018))
states that there exists a set of emotions that are
considered distinct from one another (Ekman, 1999).
These distinct emotions are labelled basic because
they are (i) deeply rooted in subcortical areas of the
brain and have been the most influenced by evolution
(Ekman and Cordaro, 2011) and (ii) are the founda-
tion for more cognitive and culturally mediated emo-
tions, such as shame or guilt. BET asserts that for
an emotion to be considered basic it should be cou-
pled with highly consistent and distinct psychologi-
cal, behavioural, and physiological patterns (Ekman
and Rosenberg, 1997).
Therefore, emotion is defined here as the subjec-
tive experience of a particular emotion coupled with
physiological and behavioural activity that occurs si-
multaneously (Ekman and Rosenberg, 1997). This
definition encapsulates both the “raw” subjective feel-
ing of each emotion and the associated activity that
researchers can objectively measure (Ekman, 1999).
PEM considers the following emotions to be the basic
emotions: Anger, Anxiety, Disgust, Fear, Joy, Sad-
ness, Surprise. Donovan et al. (2020) discuss the ra-
tionale behind this selection.
In the basic emotion literature, there is an ongoing
debate about the universality of emotions and whether
emotions have culturally independent and universal
signals (Lewis et al., 2010) (Barrett et al., 2016) (Bar-
rett, 2017) (McDuff et al., 2017). A strong position on
the universality of these emotion signals is not taken
by the authors, as the validity of PEM is not depen-
dent on the apriori assumptions of BET. PEM focuses
on the basic emotions for pragmatic purposes, as the
basic emotions are well-defined and have been thor-
oughly researched. However, PEM is seen as a tool
for providing insights into this debate. The meta-
analytical and multimodal approach provides a means
for robust analysis that can add clarity to this debate.
3 The PEM Workflow
The primary goal of the PEM workflow is to pro-
vide an end-to-end solution for personality-emotion
modelling that results in a live instance. Live in-
stances will provide (a) empirical insight into multiple
open questions surrounding Personality science; (b)
providing a high-resolution understanding of the spe-
cific emotive nature of personality traits; (c) providing
an analytical dashboard to enable a meta-analysis of
the magnitude and direction of results across research
groups. The varied nature of these insights reflects the
multi-purpose uses of the PEM workflow.
Two primary groups are envisaged as users of
PEM, namely theoretical researchers and industry
professionals. Researchers will consist mostly of psy-
chology and Affective computing practitioners. Simi-
larly, industry users will consist mostly of psycholog-
ical professionals in various domains (e.g. education)
and those interested in user-profiling (e.g. advertising
Figure 2: The PEM Workflow
15 Personality Traits Fear Anger Joy Anxiety Sadness Surprise Disgust
Openness to Experience X* X X X X X X
Openness X X* X X X X X
Intellect X X X* X X X X
Conscientiousness X X X X* X X X
Industriousness X X X X X* X X
Orderliness X X X X X X* X
Extraversion X X X X X X X*
Assertiveness X* X X X X X X
Enthusiasm X X* X X X X X
Agreeableness X X X* X X X X
Compassion X X X X* X X X
Politeness X X X X X* X X
Neuroticism X X X X X X* X
Withdrawal X X X X X X X*
Volatility X* X X X X X X
Figure 3: Analytical dashboard for visualising statistical correlation of state-to-trait mappings for a PEM live instance. Note:
the magnitude and direction indicated by this figure were generated randomly. For an example of the analytical dashboard
based on empirical research, see (Donovan et al., 2020b) and (Donovan et al., 2020c)
or political campaigning). In both cases, PEM enables
a holistic comprehension of the needs and character-
istics of target populations.
The PEM workflow consists of several stages as
depicted in Figure 2:
(i) Participant pool: A suitable participant pool,
which is dependent on the aims of the specific study,
is recruited by the user groups for data collection.
(ii) Data collection:
The PEM workflow necessitates two types of data
1. Personality data types.
The hierarchical trait model used in PEM requires
the use of data from the Big Five Aspects Scale
(BFAS). The BFAS breaks down the personality
of the participant based the Five Factors and their
sub-traits as described in Section 2.
2. Emotive data types. PEM necessitates data collec-
tion from at least one of the emotion modalities.
Data collection can either be conducted directly or
15 Personality Traits Anger Disgust Fear Joy Sadness Surprise
Openness to Experience X* X X X X X
Openness X X* X X X X
Intellect X X X* X X X
Conscientiousness X X X X* X X
Industriousness X X X X X* X
Orderliness X X X X X X*
Extraversion X* X X X X X
Assertiveness X* X X X X X
Enthusiasm X X* X X X X
Agreeableness X X X* X X X
Compassion X X X X* X X
Politeness X X X X X* X
Neuroticism X X X X X X*
Withdrawal X* X X X X X
Volatility X X* X X X X
Figure 4: Analytical dashboard for visualising statistical correlation of state-to-trait mappings for a PEM live instance. Note:
the magnitude and direction indicated by this figure were generated randomly. For an example of the analytical dashboard
based on empirical research, see (Donovan et al., 2020b) and (Donovan et al., 2020c)
taken from already existing data-sets. The modal-
ities are video capturing, written or transcribed
text, audio, and self/peer report.
(iii) Feature Extraction: Features are extracted
for emotional content based on the modality of the
data. For example, from video capturing PEM ex-
tracts facial landmarks points that indicate emotional
expressions (e.g. muscle movements around the face,
eyebrows, nose, and mouth). This process is currently
automated by a support vector machine classification
trained on two different datasets (Healy et al., 2018).
(iv) Data Analysis: Descriptive statistics are de-
rived from the underlying features to compute the
mean for each basic emotion per person. The data
is also assessed to ensure that it matches parametric
assumptions of the inferential statistical tests.
The processed emotion data is then statistically
correlated via a Pearson R correlational test with the
personality traits of the participants contributing a live
instance. Currently, the PEM method for conducting
correlations is Pearson R. Pearson R measures the lin-
ear relationship between two variables via the R effect
size, ranging from a score of -1 (indicating a complete
negative linear relationship between two variables) to
+1 (a complete positive linear relationship), with 0 in-
dicating no linear relationship (Field et al., 2012).
(v) PEM Model Instance: Figure 4 depicts an ex-
ample of the analytical dashboard of a live instance
with correlations between each of the 15 personality
traits and the 7 basic emotions.
The nature of the relationship (positive, negative,
or no relationship) and the strength of that relation-
ship is represented by the colours green, red, and
white. Green indicates a positive relationship, red in-
dicates a negative relationship, and white indicates no
relationship. Darker shades indicate a stronger rela-
tionship. This shading is based on the magnitude of
the Pearson R effect size (Cohen, 2013).
Each live instance dynamically updates the analyt-
ical dashboard as new data is inputted. This enables
live analysis of state-trait mappings in terms of statis-
tical significance, magnitude, and direction.
4 Discussion
An experimental study with 38 participants was car-
ried out to validate the PEM workflow (Donovan
et al., 2020b). The results of the study demonstrated
that 4 of the 7 basic emotions (Disgust, Joy, Sadness,
and Surprise) considered in PEM correlated signifi-
cantly and substantially with one or several person-
ality traits. Two further basic emotions approached
statistical significance (Fear and Anger) in their cor-
relations with personality traits.
Several of the significant relationships found be-
tween personality traits and basic emotions either
only (i) emerged at the sub-trait level, or (ii) the mag-
nitude of the relationship was greater at the sub-trait
level than the Five Factor level. This result sup-
ports the need for high-resolution analysis of person-
ality incorporated by PEM, as otherwise these find-
ings would have been undetected 2.
4.1 PEM’s Contribution to
Longstanding Research Questions
PEM can provide contributions to several research
questions. These research questions include:
1. What is the emotive nature of personality?
2. How to minimise the effects of context variance
in basic emotion studies?
3. What are the main emotion sequencing effects and
how do they differ with respect to individual dif-
4.1.1 What is the emotive nature of personality?
Outside of a few research studies, e.g. (Donovan
et al., 2020a) and (Gavrilescu, 2015), the relationship
between personality and emotion outside of Extraver-
sion and Neuroticism has been overlooked. This is
unfortunate, because understanding the relationship
between these two phenomena is important, as re-
search that can detect this relationship has the poten-
tial to enable greater user adaptability in both theoret-
ical and practical domains.
PEM addresses this question by explicitly mod-
elling the interaction between personality and emo-
tions across modalities. The results of PEM studies
- once pooled together into an analytical dashboard -
will enable one to draw empirically driven observa-
tions of the emotive nature of personality traits.
4.1.2 How to minimise the effects of context
variance in basic emotion studies?
The debate regarding the influence of cultural differ-
ences on the BET warrants further empirical study
and analysis. PEM contributes to this debate by (i)
enabling comparisons between real-time PEM dash-
board instances across modalities. This enables a
comparison between populations from different cul-
tures for differences in emotional expressions. PEM
also contributes to this debate by (ii) enabling a mul-
timodal perspective, enabling researchers to assess
emotions based on a broad spectrum of input, includ-
ing voice, facial expressions, language, and self/peer
reports. This contribution (ii) avoids the criticism tar-
geted at BET research, that the BET field is over-
reliant on using subjective self-report methodology
2See also (Donovan et al., 2020c). This preprint com-
pares personality to emotion mappings across the modali-
ties of self-report and facial expressions.
for collecting data (Barrett, 2017). Both contributions
represent means of either minimising and/or under-
standing the effects of culturally based variances con-
cerning basic emotions.
4.1.3 What are the main emotion sequencing
effects and how do they differ with respect
to individual differences?
A common experimental protocol in BET research is
to separate emotional stimuli with a neutral task to
bring participants back to a “baseline” state. This is
done to avoid sequencing effects, where the experi-
ence of one emotion either increases or decreases the
likelihood of experiencing subsequent emotions. For
example, the emotions Anger and Disgust have been
shown to have sequencing effects with one another,
where the experience of one makes the experience of
the other emotion more likely to occur (Salerno and
Peter-Hagene, 2013). However, it is not clear whether
there exist sequencing effects across other basic emo-
tions, and whether personality plays any mediating
role for such effects. PEM provides a platform for
researchers to design future studies to test such hy-
potheses and to assess sequencing effects with respect
to individual differences.
4.2 The Sustainability of the PEM
The sustainability and reusability of the workflow are
differentiated between the two primary user groups -
theoretical researchers (both in AC and Psychology)
and industry professionals.
PEM provides researchers with a workflow for
generating, testing, and developing applications to
improve human-computer interaction. For instance,
in semantic analysis research the challenge of distin-
guishing between emotional states in the presence of
sarcasm or jokes is a difficult one. However, per-
sonality has been successfully applied to determine
the meaning of ambiguous statements (Poria et al.,
2016). AC researchers could utilise personality data
to minimise ambiguities in content from client or user
groups (Mehta et al., 2019) resulting in higher accu-
PEM provides professionals with a workflow for
a better understanding of the emotional states or per-
sonality traits of a target audience. For example, to
gauge the personality of a target audience, text from
the audience could be collected. Semantic analysis
can then be conducted to assess the basic emotional
states of the audience, which can then be used to infer
their associated personality traits.
5 PEM Use Case: A Non
Questionnaire Method of
Personality Assessment
There are several potential applications for the PEM
workflow. One potential application is in the do-
main of personality assessment. The current method-
ology for assessing personality is the use of Likert-
scale questionnaires (Anglim and O’Connor, 2018).
These questionnaires ask participants to rate how well
statements regarding typical patterns of thought, be-
haviour, emotion, and motivation apply to themselves
or another person. The accuracy of questionnaires as
a personality assessment tool indicates that they re-
port quite well on both accounts, but they are not ex-
haustive (Vinciarelli and Mohammadi, 2014a).
There are several deficiencies with questionnaires
that inhibit an exhaustive detection of personality:
• Highly accurate personality questionnaires are
time-consuming and monotonous to complete
(e.g. the NEO-FI-R contains over 200 questions
and takes over an hour to complete on average).
Shorter questionnaires can take less than 10 min-
utes to complete, but they are significantly less ac-
curate (Anglim and O’Connor, 2018).
Questionnaires assume a level of cognitive com-
petence that effectively excludes several groups of
people (e.g. young children, people with literary
issues, and cognitive disorders).
Results also show that repeatedly taking person-
ality questionnaires decreases its accuracy for the
results of that person, a phenomenon known as a
“retest artifact” (Durham et al., 2002).
Personality questionnaires are a secondary source.
Questionnaires require participants to sit and fo-
cus on a set number of questions for a length
of time, rather than assessing personality “in the
wild” from real-world interactions (Baumeister
et al., 2007).
PEM provides a foundation for non-
questionnaire-based inferences of personality
via state-to-trait mappings. Interested parties can
acquire and analyse data via stages (ii) - (iii) to
extract information about emotion states. Users can
then use this information along with a state-to-trait
live instance (v-vi) to infer the likely personality
traits of each member of the participant pool, given
their observed emotive states.
There are several advantages to such an approach,
as it would (a) be inclusive to all members of the
population, and (b) it enables continuous personal-
ity assessment based on data acquired from multiple
modalities. The value of assessing personality via the
modalities listed in stage (i) is that such signals can
indicate personality in natural settings (e.g. semantic
analysis of text messages). This would constitute a
valuable personality assessment tool.
6 Conclusions
The Personality Emotion Model (PEM) work-
flow enables automated and quantified bi-directional
personality-emotion mappings. PEM specifies a stan-
dardised approach for data collection, specifying the
required data input for personality and emotion to en-
able mappings. PEM identifies methods of feature ex-
traction to identify the emotive content per modality
that is needed for inferential statistical analysis. The
appropriate statistical tests to correlate these phenom-
ena and to generate a model instance is also identified
in the workflow. Overall, PEM provides automation
for many of these aspects, providing a tool for extract-
ing emotive states and correlating them with person-
ality traits that are built-into the workflow.
PEM provides several advantages over modern ap-
proaches in personality/emotion research: (i) PEM
utilises a meta-analytical and data-driven approach to
quantify and reduce the impact of sampling/cultural
variance; (ii) PEM provides a methodology that facil-
itates contributions to research questions in the fields
of Personality science and Affective computing; (iii)
PEM assesses personality at a high-resolution level of
the trait hierarchy increasing the likelihood of captur-
ing statistically significant relationships between per-
sonality traits and the basic emotions; (iv) PEM fa-
cilitates multi-purpose applications of results across
research and industrial domains; (v) PEM is flexible,
enabling the model to be used in domains relating to
personality and/or emotions; (vi) PEM is adaptive, in
that the model will be refined as more data is col-
lected; (vii) The results of PEM will be open-sourced
and available online for interested groups to use.
PEM enables practical applications. One po-
tential application is the establishment of a non-
questionnaire method of personality assessment.
Such a method of personality assessment would en-
able researchers to study personality in more natural-
istic settings, using a combination of modalities to ex-
tract information and identify personality in natural
Allen, T. A., Carey, B. E., McBride, C., Bagby, R. M., DeY-
oung, C. G., and Quilty, L. C. (2018). Big Five aspects
of personality interact to predict depression. Journal
of Personality, 86(4):714–725. Publisher: John Wiley
& Sons, Ltd.
Anglim, J. and O’Connor, P. (2018). Measurement and
research using the Big Five, HEXACO, and narrow
traits: A primer for researchers and practitioners. Aus-
tralian Journal of Psychology.
Barrett, L. F. (2017). How emotions are made: The secret
life of the brain. Houghton Mifflin Harcourt.
Barrett, L. F., Lewis, M., and Haviland-Jones, J. M. (2016).
Handbook of emotions. Guilford Publications.
Baumeister, R. F., Vohs, K. D., and Funder, D. C. (2007).
Psychology as the science of self-reports and finger
movements: Whatever happened to actual behavior?
Perspectives on Psychological Science, 2(4):396–403.
Publisher: Sage Publications Sage CA: Los Angeles,
Cohen, J. (2013). Statistical power analysis for the behav-
ioral sciences. Routledge.
Costa, P. T. and McCrae, R. R. (1995). Solid ground in the
wetlands of personality: A reply to Block.
Davis, K. L. and Panksepp, J. (2018). The Emotional Foun-
dations of Personality: A Neurobiological and Evolu-
tionary Approach. WW Norton & Company.
DeYoung, C. G. (2015). Cybernetic big five theory. Journal
of Research in Personality, 56:33–58.
DeYoung, C. G., Quilty, L. C., and Peterson, J. B. (2007).
Between facets and domains: 10 aspects of the Big
Five. Journal of personality and social psychology,
Donovan, R., Cotter, J., deRoiste, A., and O’Reilly, R.
(2020a). Improving Academic Performance Amongst
First Years Computer Science Students Through Goal-
Setting. Donegal. IEEE.
Donovan, R., Johnson, A., deRoiste, A., and O’Reilly, R.
(2020b). Quantifying the Links between Personal-
ity Sub-Traits and the Basic Emotions. In Affective
Computing and Emotion Recognition, Lecture Notes
in Computer Science, Cagliari. Springer.
Donovan, R., Johnson, A., and O’Reilly, R. (2020c). Dif-
ferentiation in Personality-Emotion Mappings Self-
Reported and Automatically Extracted Emotions (pre-
Durham, C. J., McGrath, L. D., Burlingame, G. M.,
Schaalje, G. B., Lambert, M. J., and Davies, D. R.
(2002). The effects of repeated administrations on
self-report and parent-report scales. Journal of Psy-
choeducational Assessment, 20(3):240–257.
D’Mello, S., Kappas, A., and Gratch, J. (2018). The affec-
tive computing approach to affect measurement. Emo-
tion Review, 10(2):174–183. Publisher: SAGE Publi-
cations Sage UK: London, England.
Ekman, P. (1999). Basic emotions. Handbook of cognition
and emotion, pages 45–60.
Ekman, P. and Cordaro, D. (2011). What is meant by calling
emotions basic. Emotion review, 3(4):364–370.
Ekman, P. and Rosenberg, E. L. (1997). What the face
reveals: Basic and applied studies of spontaneous
expression using the Facial Action Coding System
(FACS). Oxford University Press, USA.
El Bachari, E., Abdelwahed, E., and El Adnani, M. (2010).
Design of an adaptive e-learning model based on
learner’s personality. Ubiquitous Computing and
Communication Journal, 5(3):1–8.
Field, A., Miles, J., and Field, Z. (2012). Discovering statis-
tics using R. Sage publications.
Gavrilescu, M. (2015). Study on determining the Big-Five
personality traits of an individual based on facial ex-
pressions. pages 1–6. IEEE.
Graziano, W. G. and Tobin, R. M. (2009). Agreeableness.
Healy, M., Donovan, R., Walsh, P., and Zheng, H. (2018). A
machine learning emotion detection platform to sup-
port affective well being. pages 2694–2700. IEEE.
Hirsh, J. B., DeYoung, C. G., Xu, X., and Peterson, J. B.
(2010). Compassionate liberals and polite conserva-
tives: Associations of agreeableness with political ide-
ology and moral values. Personality and Social Psy-
chology Bulletin, 36(5):655–664.
Hutto, D. D., Robertson, I., and Kirchhoff, M. D. (2018). A
new, better BET: rescuing and revising basic emotion
theory. Frontiers in psychology, 9:1217. Publisher:
John, O. P., Naumann, L. P., and Soto, C. J. (2008).
Paradigm shift to the integrative big five trait taxon-
omy. Handbook of personality: Theory and research,
Leon, F., Morales, O., and V´
ertiz, H. (2017). Person-
ality traits that differentiate Attendants of Higher-
Education Online Courses. Journal of E-Learning and
Knowledge Society, 13:141–148.
Lewis, M., Haviland-Jones, J. M., and Barrett, L. F. (2010).
Handbook of emotions. Guilford Press.
Little, B. (2014). Me, myself, and us: The science of per-
sonality and the art of well-being. Public Affairs.
McAdams, D. P. (2015). The art and science of personality
development. Guilford Publications.
McAdams, D. P. and Pals, J. L. (2006). A new big five:
Fundamental principles for an integrative science of
personality. American psychologist, 61(3):204.
McCrae, R. R. and Costa Jr, P. T. (1999). A five-factor the-
ory of personality. Handbook of personality: Theory
and research, 2(1999):139–153.
McCrae, R. R. and John, O. P. (1992). An introduction to
the five-factor model and its applications. Journal of
personality, 60(2):175–215.
McDuff, D., Girard, J. M., and El Kaliouby, R. (2017).
Large-scale observational evidence of cross-cultural
differences in facial behavior. Journal of Nonverbal
Behavior, 41(1):1–19. Publisher: Springer.
Mehl, M. R., Gosling, S. D., and Pennebaker, J. W. (2006).
Personality in its natural habitat: manifestations and
implicit folk theories of personality in daily life. Jour-
nal of personality and social psychology, 90(5):862.
Mehta, Y., Majumder, N., Gelbukh, A., and Cambria, E.
(2019). Recent trends in deep learning based person-
ality detection. Artificial Intelligence Review, pages
1–27. Publisher: Springer.
Picard, R. W. (2000). Affective computing. MIT press.
Poria, S., Cambria, E., Bajpai, R., and Hussain, A. (2017).
A review of affective computing: From unimodal
analysis to multimodal fusion. Information Fusion,
37:98–125. Publisher: Elsevier.
Poria, S., Cambria, E., Hazarika, D., and Vij, P.
(2016). A deeper look into sarcastic tweets using
deep convolutional neural networks. arXiv preprint
Revelle, W. and Scherer, K. R. (2004). Personality and emo-
Salerno, J. M. and Peter-Hagene, L. C. (2013). The in-
teractive effect of anger and disgust on moral outrage
and judgments. Psychological Science, 24(10):2069–
Soto, C. J. (2019). How replicable are links between person-
ality traits and consequential life outcomes? The Life
Outcomes of Personality Replication Project. Psycho-
logical Science, 30(5):711–727.
Soto, C. J. and John, O. P. (2017). The next big five inven-
tory (bfi-2): Developing and assessing a hierarchical
model with 15 facets to enhance bandwidth, fidelity,
and predictive power. Journal of personality and so-
cial psychology, 113(1):117.
Subramanian, R., Wache, J., Abadi, M. K., Vieriu, R. L.,
Winkler, S., and Sebe, N. (2016). ASCERTAIN:
Emotion and personality recognition using commer-
cial sensors. IEEE Transactions on Affective Comput-
ing, 9(2):147–160. Publisher: IEEE.
Vinciarelli, A. and Mohammadi, G. (2014a). More person-
ality in personality computing. IEEE Transactions on
Affective Computing, 5(3):297–300.
Vinciarelli, A. and Mohammadi, G. (2014b). A survey of
personality computing. IEEE Transactions on Affec-
tive Computing, 5(3):273–291.
Wilson, R. E., Thompson, R. J., and Vazire, S. (2017). Are
fluctuations in personality states more than fluctua-
tions in affect? Journal of Research in Personality,
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
Recently, the automatic prediction of personality traits has received a lot of attention. Specifically, personality trait prediction from multimodal data has emerged as a hot topic within the field of affective computing. In this paper, we review significant machine learning models which have been employed for personality detection, with an emphasis on deep learning-based methods. This review paper provides an overview of the most popular approaches to automated personality detection, various computational datasets, its industrial applications, and state-of-the-art machine learning models for personality detection with specific focus on multimodal approaches. Personality detection is a very broad and diverse topic: this survey only focuses on computational approaches and leaves out psychological studies on personality detection.
Conference Paper
Full-text available
This paper describes a new emotional detection system based on a video feed in real-time. It demonstrates how a bespoke machine learning support vector machine (SVM) can be utilized to provide quick and reliable classification. Features used in the study are 68-point facial landmarks. In a lab setting, the application has been trained to detect six different emotions by monitoring changes in facial expressions. Its utility as a basis for evaluating the emotional condition of people in situations using video and machine learning is discussed.
Full-text available
The Big Five personality traits have been linked with dozens of life outcomes. However, metascientific research has raised questions about the replicability of behavioral science. The Life Outcomes Of Personality Replication (LOOPR) Project was therefore conducted to estimate the replicability of the personality-outcome literature. Specifically, we conducted preregistered, high-powered (median N = 1,504) replications of 78 previously published trait-outcome associations. Overall, 87% of the replication attempts were statistically significant in the expected direction. The replication effects were typically 77% as strong as the corresponding original effects, which represents a significant decline in effect size. The replicability of individual effects was predicted by the effect size and design of the original study, as well as the sample size and statistical power of the replication. These results indicate that the personality-outcome literature provides a reasonably accurate map of trait-outcome associations, but also stands to benefit from efforts to improve replicability.
Full-text available
Basic Emotion Theory, or BET, has dominated the affective sciences for decades (Ekman, 1972, 1992, 1999; Ekman and Davidson, 1994; Griffiths, 2013; Scarantino and Griffiths, 2011). It has been highly influential, driving a number of empirical lines of research (e.g., in the context of facial expression detection, neuroimaging studies and evolutionary psychology). Nevertheless, BET has been criticized by philosophers, leading to calls for it to be jettisoned entirely (Colombetti, 2014; Hufendiek, 2016). This paper defuses those criticisms. In addition, it shows that we have good reason to retain BET. Finally, it reviews and puts to rest worries that BET’s commitment to affect programs renders it outmoded. We propose that, with minor adjustments, BET can avoid such criticisms when conceived under a radically enactive account of emotions. Thus, rather than leaving BET behind, we show how its basic ideas can be revised, refashioned and preserved. Hence, we conclude, our new BET is still a good bet.
Full-text available
Objective Personality traits influence human behaviour across a broad range of situations and are consequently relevant to many theoretical and applied disciplines. In this perspective piece, we provide an overview of the logic underpinning personality measurement and review major personality taxonomies. We provide an extensive set of recommendations for researchers and practitioners on when and how to use measures of personality traits. Method We overview a range of taxonomic representations of personality structure focusing particularly on hierarchical representations and five and six factor models such as the Big Five and HEXACO models. We review the various strengths and weaknesses of each approach. Results The review outlines the major reasons for the dominance of the Big Five model, and suggests it is a good descriptive framework for studying personality in general. However we suggest that researchers and practitioners also consider alternative taxonomic personality representations such as the HEXACO. We provide a range of scenarios whereby alternative frameworks will be more appropriate than the Big Five and offer recommendations both for choosing measures in general and for implementing studies examining personality facets. Conclusion Whilst the Big Five represents an excellent general personality framework that is appropriate across multiple situations, researchers and practitioners should be aware of alternative measures and utilise them where appropriate.
Full-text available
Objective: Research has shown that three personality traits-neuroticism, extraversion, and conscientiousness-moderate one another in a three-way interaction that predicts depressive symptoms in healthy populations. We test the hypothesis that this effect is driven by three-lower order traits: withdrawal, industriousness, and enthusiasm. We then replicate this interaction within a clinical population for the first time. Method: Sample 1 included 376 healthy adults. Sample 2 included 354 patients diagnosed with current major depressive disorder. Personality and depressive tendencies were assessed via the Big Five Aspect Scales and Personality Inventory for DSM-5 in Sample 1, respectively, and by the NEO-PI-R and Beck Depression Inventory-II in Sample 2. Results: Withdrawal, industriousness, and enthusiasm interacted to predict depressive tendencies in both samples. The pattern of the interaction supported a "best two out of three" principle, in which low risk scores on two trait dimensions protects against a high risk score on the third trait. Evidence was also present for a "worst two out of three" principle, in which high risk scores on two traits are associated with equivalent depressive severity as high risk scores on all three traits. Conclusions: These results highlight the importance of examining interactive effects of personality traits on psychopathology. This article is protected by copyright. All rights reserved.
Full-text available
This study utilized the ten aspects of the Big Five personality system to detect differences in personality traits between attendants of higher education online courses and attendants of higher education face-to-face courses. Distance education theoreticians have formulated hypotheses on the personality traits that make the Autonomy, Connectedness, Diversity, and Openness attributes of the connectivist learning model attractive. 753 participants in executive extension courses of a Peruvian university in Lima (mean age = 34.62 years) filled-in an online questionnaire which included the Big Five Aspect Scales. Whereas none of the Big Five factors discriminated between attendants and non-attendants of online courses, the former emerged as significantly higher in Industriousness (an aspect of Conscientiousness) and lower in Orderliness (the second aspect of Conscientiousness) and Enthusiasm (an aspect of Extraversion) than the latter. Theoretical and practical implications of the findings are addressed and the conduct of confirmatory studies outside Peru and involving MOOCs is recommended.
Affective computing (AC) adopts a computational approach to study affect. We highlight the AC approach towards automated affect measures that jointly model machine-readable physiological/behavioral signals with affect estimates as reported by humans or experimentally elicited. We describe the conceptual and computational foundations of the approach followed by two case studies: one on discrimination between genuine and faked expressions of pain in the lab, and the second on measuring nonbasic affect in the wild. We discuss applications of the measures, analyze measurement accuracy and generalizability, and highlight advances afforded by computational tipping points, such as big data, wearable sensing, crowdsourcing, and deep learning. We conclude by advocating for increasing synergies between AC and affective science and offer suggestions toward that direction.
Affective computing is an emerging interdisciplinary research field bringing together researchers and practitioners from various fields, ranging from artificial intelligence, natural language processing, to cognitive and social sciences. With the proliferation of videos posted online (e.g., on YouTube, Facebook, Twitter) for product reviews, movie reviews, political views, and more, affective computing research has increasingly evolved from conventional unimodal analysis to more complex forms of multimodal analysis. This is the primary motivation behind our first of its kind, comprehensive literature review of the diverse field of affective computing. Furthermore, existing literature surveys lack a detailed discussion of state of the art in multimodal affect analysis frameworks, which this review aims to address. Multimodality is defined by the presence of more than one modality or channel, e.g., visual, audio, text, gestures, and eye gage. In this paper, we focus mainly on the use of audio, visual and text information for multimodal affect analysis, since around 90% of the relevant literature appears to cover these three modalities. Following an overview of different techniques for unimodal affect analysis, we outline existing methods for fusing information from different modalities. As part of this review, we carry out an extensive study of different categories of state-of-the-art fusion techniques, followed by a critical analysis of potential performance improvements with multimodal analysis compared to unimodal analysis. A comprehensive overview of these two complementary fields aims to form the building blocks for readers, to better understand this challenging and exciting research field.