Conference PaperPDF Available

Assessment of the Emotional State by Psycho-physiological and Implicit Measurements

Authors:

Abstract

The ongoing assessment in a digital educational game is a prerequisite for providing in-game adaptations aiming to enhance the learner´s current emotional state. Our assessment procedure combines the results of two modalities: i) by interpreting the learner´s interactions with the virtual environment, and ii) by psychophysiological patterns covered by facial electromyography and electrodermal activity. The combination of signals and indicators is described separately for each modality and completed by details on the interpretation of the results, based on intrapersonal comparisons. Finally, we outline how to integrate the results of both modalities into a single value that indicates the current emotional state. We conclude with a short description of how we will validate the overall assessment procedure
Assessment of the Emotional State by Psycho-
physiological and Implicit Measurements
Michael A. Bedek1
michael.bedek@tugraz.at
Ben Cowley2
ben.cowley@aalto.fi
Paul Seitlinger1
paul.seitlinger@tugraz.at
Martino Fantato2
martino.fantato@aalto.fi
Simone Kopeinik1
simone.kopeinik@tugraz.at
Dietrich Albert1
dietrich.albert@tugraz.at
Niklas Ravaja2,3
niklas.ravaja@aalto.fi
1Knowledge Management Institute
Graz University of Technology
Brückenkopfgasse 1/6, 8020 Graz
(+43) 3168739553
2Center for Knowledge & Innovation
Research
Aalto Yliopisto, Helsinki
PO Box 21250, 00075 Aalto
(+358) 505631249
3Department of Social Research
University of Helsinki
PO Box 54, 000 14 University of
Helsinki
(+358) 503605191
ABSTRACT
The ongoing assessment in a digital educational game is a
prerequisite for providing in-game adaptations aiming to
enhance the learner´s current emotional state. Our assessment
procedure combines the results of two modalities: i) by
interpreting the learner´s interactions with the virtual
environment, and ii) by psychophysiological patterns covered by
facial electromyography and electrodermal activity. The
combination of signals and indicators is described separately for
each modality and completed by details on the interpretation of
the results, based on intrapersonal comparisons. Finally, we
outline how to integrate the results of both modalities into a
single value that indicates the current emotional state. We
conclude with a short description of how we will validate the
overall assessment procedure.
Categories and Subject Descriptors
K.8.0 [PERSONAL COMPUTING]: General games.
General Terms
Measurement, Human Factors, Theory.
Keywords
Psychophysiological Measurements, Implicit Measurements,
Emotion, State.
1. INTRODUCTION
The TARGET project aims to develop a new technology
enhanced learning (TEL) platform to support competence
development in the learning domains of project and innovation
management and global sustainable manufacturing
(http://www.reachyourtarget.org). The major component of the
platform is a digital educational game (DEG). This DEG
consists of game scenarios possessing critical incidents from the
learning domains in order to support the knowledge transfer
between the virtual world and the learner`s ordinary working
life. Within this DEG, the learner is represented by an avatar and
interacts with several non-playable characters (NPCs). It is
necessary to communicate with different NPCs and to gather
pieces of information in order to master the scenario. The
learner may use several buttons on the keyboard which have the
effect that the avatar expresses specific emotions by facial
expressions or body movements. Additionally, several tools are
available, for example a chat tool to communicate with the
NPCs, a teleport tool to switch between different locations or a
face cam which shows the avatar`s own face to enhance
awareness of currently expressed emotions.
Even if a given DEG possesses the potential to provide a
learning context which is intrinsically motivating and
emotionally appealing to engage with, a DEG which adapts to
the learner´s current state is expected to increase the probability
that this engagement potential is realised. In TARGET, our aim
is to provide in-game adaptations [7] tailored to optimize the
learner´s current achievement motivation, cognitive workload,
problem representation and emotional state for an efficient and
sustainable learning process. The prerequisite for appropriate
adaptations is a valid assessment of these constructs.
According to [2] emotion and related constructs encompass
three aspects: i) a subjective cognitive experience (e.g. assessed
by questionnaires), ii) behavioural expressions (i.e. actions and
behavioural patterns assessed by implicit techniques) and iii)
psychophysiological patterns. Since an explicit assessment by
means of a short questionnaire appearing in regular time
intervals would most likely disturb the learner`s flow experience
[3], we aim to continuously assess the latter two aspects:
behaviour and psychophysiology.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
ICMI11, November 1418, 2011, Alicante, Spain
Copyright 2011 ACM 1-58113-000-0/00/0010…$10.00.
Such a multi-modal assessment depends on the two modalities
being able to share a common model of a given construct, or
else translate between different models that purport to describe
the same construct. We apply the circumplex model of emotion
proposed by [9], which consists of the two independent
dimensions of pleasantness and activation. Pleasantness is a
continuous bipolar dimension between pleasant and unpleasant.
Activation, also called arousal, is a continuous unipolar
dimension with the poles low and high activation (arousal).
This paper focuses on the description of two modalities to assess
the learner‟s emotional state. The first modality is based on
modelling and interpreting the learner´s interactions with the
virtual environment via keyboard and mouse. The second
modality uses psychophysiological signals which have been
empirically linked to emotional states. We will describe the
signals and indicators for both modalities, their theoretical basis
and how they are combined and interpreted. This is a work-in-
progress and as evaluation is on-going the paper will remain at a
schematic level with no results section. However, we conclude
by outlining how we will validate the assessment of both
modalities.
2. MEASUREMENTS
In the following, we describe the different indicators and signals
separated for both sources, starting with the implicit assessment
technique.
2.1 Behavioural Indicators
The implicit behavioral indicators are based on the
interpretation of a learner´s actions and interactions within the
virtual environment [1]. A set of behavioural indicators for the
assessment of emotion as state is provided in table 1. Some of
them are primarily related to activation, such as click rate (#1)
or inactivity (#9). Others are primarily related to pleasantness,
such as using defined keys to express positive or negative
emotions (#7 and #8). In order to continuously compute each
indicator, the game play is divided into consecutive and equally
long time slices, lasting for e.g. 30 seconds. All indicators are
operationalized and described in detail in [1]. The indicators 10
to 14 are directly derived from the theory of information
foraging [12], which describes the strategies that people apply to
search and gather information.
Table 1. Set of Behavioural Indicators
#
Behavioural Indicator
1
Click rate
2
Length of mouse movements
3
Relative exploitation of available tools
4
Frequency of tool-usage (of each available tool)
5
Frequency of communication tool-usage
6
Frequency of interactions with NPCs
7
Frequency of expressing positive emotions via keys
8
Frequency of expressing negative emotions via keys
9
Inactivity [sec.]
10
Within-patch processing [sec.]
11
Between-patch Processing [sec.]
12
Extent of NPC-interactions weighted by amount of
Within-Patch processing
13
Information gained
14
Rate of information gain
In the theory of information foraging, human search behaviour
is regarded as adaptive to the environment to gain information
from external sources as effectively and efficiently as possible.
External sources are called patches (e.g. online documents). An
ideal information forager maximizes the rate of gaining valuable
information by seeking for a balanced ratio between explorative
and exploitative search behaviour. Available time needs to be
divided into the search for new sources bearing valuable
information (Between-Patch processing) as well as into
elaborated processing of these patches to extract relevant
information (Within-Patch processing). By concentrating solely
on one single patch (e.g. a single paper) valuable information
from external resources does not become available.
Contrariwise, solely explorative search will lead to ignorance of
important details.
Even if the theory of information foraging has been initially
developed in the context of navigation on the web, we apply the
principles and adapt some of the indicators to the area of games
because of two reasons: i) the learner has to search for and to
communicate with several NPCs in order to collect all
information necessary to master the game scenarios and ii) it is
assumed that a successful information forager experiences a
positive emotional state more often than an unsuccessful one.
2.2 Psychophysiological Signals
For a thorough review of psychophysiological methods for
game-based experiments see [8]. Such methods are useful for
objectively examining game experiences, because the
physiological processes measured are mostly unwilled. Thus
these measures have several advantages over traditional self-
report: (i) measurements can be performed continuously with
high temporal resolution; (ii) processes of interest can be
covertly assessed; and (iii) these measures provide information
on emotional and motivational processes that are not available
to conscious awareness [13].
Facial electromyography (EMG) provides a direct measure of
the electrical activity associated with facial muscle contractions
that are an important form of emotional expression [15]. A
number of studies have shown that the processing of unpleasant
emotions is associated with greater activity over the corrugator
supercilii (brow) muscle region and that processing pleasant
emotions prompts greater activity over the zygomaticus major
(cheek) muscle region [10][14][18]. In addition, increased
activity at the orbicularis oculi (periocular) muscle area is
involved in the expression of enjoyment smile and genuine
pleasure [5]. Several studies have shown that orbicularis oculi
activity is particularly high during pleasurable high-arousal
emotions [14][18].
Electrodermal activity (EDA), commonly known as skin
conductance, is an important physiological index of arousal and
is innervated entirely by the sympathetic nervous system [4].
Several studies using the picture-viewing paradigm have shown
that EDA is highly correlated with self-reported emotional
arousal [10]. That is, arousing pictures of either valence result in
increased EDA as compared to low-arousal pictures.
In the laboratory and in deployment, these signals are measured
by the Varioport-ARM mobile psychophysiological data
acquisition system (Becker Meditec, Karlsruhe, Germany). This
device is lightweight and battery operated, and thus, ideal for
deployment on-site with clients.
3. INTEGRATION & INTERPRETATION
In the following, we describe how to combine the different
signals and indicators for each modality, how to interpret the
results in terms of high and low values and finally, how to
combine the results of both modalities.
3.1 Integrating Measurements per Modality
The reason for integrating the measurements for both modalities
separately in a first step is that not every learner may have
access to the Varioport device. In consequence, the validity of
each assessment technique has to be evaluated independently.
For the implicit assessment technique we will apply a multiple
regression model as suggested by [11]. A linear model would be
the simplest case; however, we will evaluate whether it delivers
better results (i.e. if it explains more variance) than other models
(such as a Generalized Linear Model which can incorporate
non-linear covariates in the coefficients). For each time slice and
both dimensions of the emotional model, i.e. activation and
pleasantness, a regression equation in the following form has to
be calculated:
xi = di + w1i * BI1i + ... + wji * BIji + ... + w14i * BI14i (1),
with xi as the initial value for the dimension i in the given time
slice, di as the constant intercept, BIji as the “raw-values” for the
14 predictors (i.e. the behavioural indicators) and finally, wji as
the predictors‟ weights. The intercepts and weights for both
regression equations are to be conducted by means of a
validation study which is briefly outlined in section 4.
The integration of psychophysiological signals involves finding
suitable functions to characterise all signals on the same scale so
that they are comparable. Both EDA and EMG transforms begin
by pre-processing. A low pass Butterworth filter is applied
where computational power allows (i.e. when the requisite
frequency domain transform can be performed by a hardware
routine this is available on the Varioport device, but may not
be on others).
3.2 Interpretation
The interpretation of the current emotional state as indicated by
the initial values xi in terms of an optimal or suboptimal state
depends on their comparison with a baseline. The baseline can
be conducted either by the learner´s values in previous time-
slices of the same game scenario (intrapersonal comparison) or
by the values of other learners. The latter approach is feasible
when an extensive database for interpersonal comparison is
available. However, we prefer an intrapersonal comparison
which takes the learner´s gaming history into account since
individual learner`s baselines may differ to a great extent.
For the implicit assessment technique we standardize both initial
values by a z-transformation:
zi = ( xi Mi ) / SDi (2),
with zi and xi as standardized and initial values of the dimension
i, respectively. The average values of the dimension i
represented by Mi is computed by averaging the initial values xi
of all previous time-slices. Thus, Mi does not incorporate the
current time-slice. The standard deviation of initial values of all
previous time slices is indicated by SDi. Since the reliability of
the standard deviation depends on the amount of incorporated
data, the deviation of the current time-slice from the averaged
previous time-slices in terms of standard deviations is not taken
into account until the fourth time-slice has passed. Hence, the
computation of the standardized values begins 120 seconds after
the learner starts to play the scenario.
Finally, in order to gather manifestations of a continuous
variable, whose values range from 0 and 1, the standardized
value zi is inserted into the following logistic function:
p(zi) = 1 / ( 1 + e-zi ) (3),
with p(zi) indicating the probabilistic value of the dimension i.
The logistic function is positively accelerated and differentiates
primarily in a range between -3 and +3.
With regards to the psychophysiological modality, after
applying the low pass Butterworth filter, EDA continues by
splitting the signal into tonic and phasic components. Tonic
values are obtained at each time t by estimating the mean of the
signal over the previous 8 seconds (8 is a standard value
reflecting the speed of EDA reactivity). Phasic values are then
obtained by subtracting the tonic from the overall at time t. The
EMG transformation proceeds by rectification.
Thereafter, for each signal, our relative approach is based on
comparing a current value at time t to the average from a time-
slice of n previous values. The size of a change which
constitutes an interesting variation (as opposed to minor
fluctuations in the signal), and thus reflects actual change in the
player‟s underlying psychological state, is Δ. The method of
indexing psychophysiological signals involves determining a
general value for n and for Δ, for each signal. Currently, the
learning part of our approach is offline so that a learned model
will be fixed for every use of the system. Offline learning
involves statistically estimating the overall fit of each
combination of n and Δ values using General Estimating
Equations on a dataset of live recordings from presentations of
emotionally evocative stimuli to subjects.
With continuous psychophysiological data for both emotional
dimensions (i.e., pleasantness and arousal), it is possible to
combine these data to „locate‟ the subject more specifically in
emotional state space. That is, a stressed emotional state is
characterized by a combination of displeasure (e.g., high
corrugator EMG activity) and high arousal (e.g., high EDA); a
positively excited state is characterized by a combination of
pleasure (e.g., high zygomatic EMG activity) and high arousal
(e.g., high EDA); depression/boredom is characterized by
combined displeasure (e.g., high corrugator EMG activity) and
low arousal (e.g., low EDA); and positive relaxation is a
combination of pleasure (e.g., high zygomatic EMG activity)
and low arousal (e.g., low EDA).
This is to be interpreted as a probable tendency toward the
stated emotion, rather than an exact emotional state per se.
Probability can be measured by the distance of the combined
signals from their theoretical maximum.
3.3 Integrating Results of both Modalities
The procedure to integrate the results of both modalities is
similar to the multiple regression equation (1) above. The
difference is that there would not be a constant intercept and
only two pairs of predictors and appropriate weights (one pair
for each modality). The weights will be fixed by the amount of
variance of an external criterion the predictor explains relative to
the amount of shared variance R2 of both predictors and the
external criterion. Put simply, the higher the predictive validity
of a modality (predictor), the higher its weight and contribution
to the final assessment result.
4. OUTLOOK: EVALUATION
For the evaluation of the indicators‟ validity we adopt
approaches suggested by [6] and [16] in order to have a non-
invasive measurement procedure and to elicit (as a dependent
measure) the third remaining part of the emotional trinity, i.e.
the subjective cognitive experience. This aspect of an emotional
state can be measured by self-report and will be used as an
external criterion to be compared with values of the
physiological and behavioral indicators.
The self-report will be mediated by a pop-up screen
intermittently occurring during game-play and displaying small
sets of (at most four) items about the current emotional state,
which can be taken for example from the PANAS Scale [17].
The learner can respond through slider scales: by moving the
position of a slider between two poles of a graphical intensity-
dimension, the learner indicates the extent to which she or he
respectively agrees or disagrees on a particular item. The
different questions or items will cover different aspects of the
two emotional dimensions and will be selected randomly for
presentation.
Finally, a regression analysis will be conducted to determine the
nature and significance of the relationship between the
indicators and the self-report. Standardized Beta-Coefficients
will support the identification of valid indicators as well as their
weights for equation (1). Additionally, the correlation of each
modality with the external criteria delivers the basis for equation
(2) which accounts for a substantial amount of emotional
variance. The regression equation can be cross-validated by
comparing the predicted with the empirical results derived from
an additional sample.
5. ACKNOWLEDGMENTS
This paper is part of the EC-Project TARGET funded by the
Seventh Framework Programme of the European Commission
(ICT-231717). The authors are solely responsible for the content
of this paper.
6. REFERENCES
[1] Bedek, M. A., Seitlinger, P., Kopeinik, S. and Albert, D.
Multivariate Assessment of Motivation and Emotion in
Digital Educational Games. 5th European Conference on
Games Based Learning, (Athens, Greece, 2011), in press.
[2] Bradley, M. M. and Lang, P. Measuring emotion: behavior,
feeling and physiology. in Lane, R. and Nadel, L. eds. The
Cognitive Neuroscience of Emotion, Oxford University
Press, New York, 2000, 242-276.
[3] Csikszentmihalyi, M. Flow: The psychology of optimal
experience. Harper & Collins, New York, 1990.
[4] Dawson, M. E., Schell, A. M. and Filion, D. L. The
electrodermal system. Cambridge University Press,
Cambridge, 2000.
[5] Ekman, P., Davidson, R. J. and Friesen, W. V. 1990. The
Duchenne smile: emotional expression and brain
physiology. II. J Pers Soc Psychol, 58 (2), 342-353.
[6] Insko, B. E. Measuring Presence: Subjective, behavioral
and physiological methods. in Riva, G., Davide, F. and
Ijsselsteijn, W. A. eds. Being There: Concepts, effects and
measurement of user presence in synthetic environments,
IOS Press, Amsterdam, 2003, 110-118.
[7] Kickmeier-Rust, M. D. and Albert, D. 2010. Micro
Adaptivity: Protecting Immersion in Didactically Adaptive
Digital Educational Games. J Comp Assist, 26 (2), 95-105.
[8] Kivikangas, J. M., Chanel, G., Cowley, B., Ekman, I.,
Salminen, M., Järvelä, S. and Ravaja, N. 2011, in press. A
review on the use of psychophysiological methods in game
research. Journal of Gaming and Virtual Worlds.
[9] Larsen, R. J. and Diener, E. Promises and problems with
the circumplex model of emotion. in Clark, M. S. ed.
Review of personality and social psychology: Emotion,
Sage, Newbury Park, 1992, 25-59.
[10] Lang, P. J., Greenwald, M. K., Bradley, M. M. and Hamm,
A. O. 1993. Looking at pictures: affective, facial, visceral,
and behavioral reactions. Psychophysiology, 30 (3), 261-
273.
[11] Margolis, M. J. and Clauser, B. E. A regression-based
procedure for automated scoring of a complex medical
performance assessment. in Williamson, D.W. Mislevy, R.
J. and Bejar, I. I. eds. Automated scoring of complex tasks
in computer-based testing, Lawrence Erlbaum Assoc,
Mahawah, 2006, 123-168.
[12] Pirolli, P. and Card, S. K. 1999. Information Foraging.
Psychol Rev, 106 (4). 643-675.
[13] Ravaja, N. 2004. Contributions of Psychophysiology to
Media Research: Review and Recommendations. Media
Psychol, 6(2), 193-235.
[14] Ravaja, N., Saari, T., Kallinen, K. and Laarni, J. 2006. The
Role of Mood in the Processing of Media Messages From a
Small Screen: Effects on Subjective and Physiological
Responses. Media Psychol, 8(3), 239-265.
[15] Tassinary, L. G., & Cacioppo, J. T. The skeletomotor
system: Surface electromyography. in J. T. Cacioppo, L. G.
Tassinary & G. G. Berntson eds. Handbook of
psychophysiology, Cambridge University Press,
Cambridge, 2000, 163199.
[16] Van Reekum, C. M., Johnstone, T., Banse, R., Etter, A.,
Wehrle, T. and Scherer, K. R. 2004. Psychophysiological
responses to appraisal dimensions in a computer game.
Cognition and Emotion, 18(5), 663-688.
[17] Watson, D., Clark, L. A. & Tellegen, A. 1988.
Development and validation of brief measures of positive
and negative affect: The PANAS Scale. J Pers Soc Psychol,
54, 1063-1070.
[18] Witvliet, C. V. and Vrana, S. R. 1995. Psychophysiological
responses as indices of affective dimensions.
Psychophysiology, 32(5), 436-443.
... However the specification of games themselves was lacking in detail. Another method, similar to what we propose here, has been demonstrated for the domain of educational games [39], [40]; here players were modelled in terms of the competences they show in the pedagogical domain. Competence model validity is improved by theorydriven expert design of features, because the psychological underpinnings of real-world competences are defined in the literature. ...
Preprint
Full-text available
We present the first in a series of three academic essays which deal with the question of how to build a generalized player model. We begin with a proposition: a general model of players requires parameters for the subjective experience of play, including at least: player psychology, game structure, and actions of play. Based on this proposition, we pose three linked research questions, which make incomplete progress toward a generalised player model: RQ1 what is a necessary and sufficient foundation to a general player model?; RQ2 can such a foundation improve performance of a computational intelligence-based player model?; and RQ3 can such a player model improve efficacy of adaptive artificial intelligence in games? We set out the arguments behind these research questions in each of the three essays, presented as three preprints. The first essay, in this preprint, reviews the literature for the core foundations for a general player model. We then propose a plan for future work to systematically extend the review and thus provide an empirical answer to RQ1 above. This work will directly support the proposed approach to address RQ2 and RQ3 above. This review was developed to support our 'Behavlets' approach to player modelling; therefore if citing this work, please use the relevant citation: Cowley B, Charles D. Behavlets: a Method for Practical Player Modelling using Psychology-Based Player Traits and Domain Specific Features. User Modelling and User-Adapted Interaction. 2016 Feb 8; online (Special Issue on Personality in Personalized Systems):1-50.
... First, homomorphic mapping means that with rich data sets on both play and game sides, it may not be necessary to compress the data space to over-simplified categories. Second, multiple solutions exist: Because the patterns of play and reaction are represented by game log activity and psychophysiology, it is valid to assume that multiple domains will be used to learn the same concepts (Bedek et al., 2011). This not only makes the whole approach more robust, but also facilitates a form of cross-validation. ...
Article
Full-text available
We report research on player modeling using psychophysiology and machine learning, conducted through interdisciplinary collaboration between researchers of computer science, psychology, and game design at Aalto University, Helsinki. First, we propose the Play Patterns And eXperience (PPAX) framework to connect three levels of game experience that previously had remained largely unconnected: game design patterns, the interplay of game context with player personality or tendencies, and state-of-the-art measures of experience (both subjective and non-subjective). Second, we describe our methodology for using machine learning to categorize game events to reveal corresponding patterns, culminating in an example experiment. We discuss the relation between automatically detected event clusters and game design patterns, and provide indications on how to incorporate personality profiles of players in the analysis. This novel interdisciplinary collaboration combines basic psychophysiology research with game design patterns and machine learning, and generates new knowledge about the interplay between game experience and design.
... While EEG itself indexes several informative features of player experience (Nacke et al., 2010), others have also found evidence that the fusion of several physiological modalities increases emotion recognition accuracy (Chanel et al., 2011) -offering the potential to apply such methods in TEL contexts. Such systems have been developed for the TARGET project (Bedek et al., 2011) and thus early work suggests their applicability also to learning games. ...
Article
Full-text available
We report an evaluation study for a novel learning platform, motivated by the growing need for serious game assessment methods. The method was a laboratory experiment combining evaluation methods from the fields of learning assessment and psychophysiology. 15 participants used the TARGET game platform for 25 minutes, while biosignals ECG, EDA and fEMG were recorded. Learning was scored using pre- and post-test question-based assessments Repeated-measures analysis with Generalised Estimating Equations was used to predict scores by tonic psychophysiological data. Results indicate some learning effect, plus a relationship between mental workload (indexed by ECG) and learning. We conclude that the best quality of insight is afforded by the combination of subjective self-report and objective psychophysiology.
Article
Full-text available
In recent years, various recommendation algorithms have been proposed to support learners in technology-enhanced learning environments. Such algorithms have proven to be quite effective in big-data learning settings (massive open online courses), yet successful applications in other informal and formal learning settings are rare. Common challenges include data sparsity, the lack of sufficiently flexible learner and domain models, and the difficulty of including pedagogical goals into recommendation strategies. Computational models of human cognition and learning are, in principle, well positioned to help meet these challenges, yet the effectiveness of cognitive models in educational recommender systems remains poorly understood to this date. This thesis contributes to this strand of research by investigating i) two cognitive learner models (CbKST and SUSTAIN) for resource recommendations that qualify for sparse user data by following theory-driven top down approaches, and ii) two tag recommendation strategies based on models of human cognition (BLL and MINERVA2) that support the creation of learning content meta-data. The results of four online and offline experiments in different learning contexts indicate that a recommendation approach based on the CbKST, a well-founded structural model of knowledge representation, can improve the users? perceived learning experience in formal learning settings. In informal settings, SUSTAIN, a human category learning model, is shown to succeed in representing dynamic, interest based learning interactions and to improve Collaborative Filtering for resource recommendations. The investigation of the two proposed tag recommender strategies underlined their ability to generate accurate suggestions (BLL) and in collaborative settings, their potential to promote the development of shared vocabulary (MINERVA2). This thesis shows that the application of computational models of human cognition holds promise for the design of recommender mechanisms and, at the same time, for gaining a deeper understanding of interaction dynamics in virtual learning systems.
Article
Full-text available
Information foraging theory is an approach to understanding how strategies and technologies for information seeking, gathering, and consumption are adapted to the flux of information in the environment. The theory assumes that people, when possible, will modify their strategies or the structure of the environment to maximize their rate of gaining valuable information. The theory is developed by (a) adaptation (rational) analysis of information foraging problems and (b) a detailed process model (adaptive control of thought in information foraging [ACT-IF]). The adaptation analysis develops (a) information patch models, which deal with time allocation and information filtering and enrichment activities in environments in which information is encountered in clusters; (b) information scent models, which address the identification of information value from proximal cues; and (c) information diet models, which address decisions about the selection and pursuit of information items. ACT-IF is instantiated as a production system model of people interacting with complex information technology. (PsycINFO Database Record (c) 2009 APA, all rights reserved)
Article
Digital educational games (DEGs) possess the potential to provide an appealing learning context which is intrinsically motivating for learners to engage with. However, this potential is either taken for granted or examined by means of questionnaires, interviews or behavioral observations in the course of evaluation studies. An adaptive game could increase the probability that a DEG is motivating and emotionally appealing. In order to adapt the game to the learner's motivational and emotional state while engaged with a particular scenario, an ongoing assessment of these states is required. However, it would probably destroy the flow-experience and the feeling of virtual presence if a questionnaire occurs repeatedly in short time intervals on the screen. Thus, it is necessary to apply an approach that assesses the motivational and emotional state in a non-intrusive way. We describe a non-intrusive assessment procedure based on the observation of behavioral indicators which might deliver evidence for the learner's states. A substantial set of behavioral indicators has been elaborated whereby some of them are derived from information foraging theory (Pirolli and Card, 1999). For example, the relative amount of time the learner is exploring the virtual environment can be considered as between-patch processing while the relative amount of time the learner is communicating with other game characters can be considered as within-patch processing. Values for each behavioral indicator (e.g. amount, frequency, seconds, etc.) are gathered repeatedly after predefined time slices, for example every 30 seconds. Afterwards, these values are contributing as weighted predictors to multiple regression equations on particular factors of a motivation model, an emotion model and a construct called clearness which is defined as appropriate problem representation. The underlying motivation model is based on the two factors of approach and avoidance motivation, the emotion model includes the factors valence and activation. A comparison of the resulting values for the constructs between the current and past time slices covers potential changes of the learner's state over time. The assessment of such changes forms the prerequisite for providing on-line game adaptations which aim to enhance the learner's state, targeting towards a full exploitation of DEGs' pedagogical potential.
Article
In this work ,we focus ,on demonstrating, a real ,time communication interface which enhances text communication by detecting from real time typed text, the extracted emotions, and displaying on the screen appropriate facial expression images ,in real time. The displayed expressions are represented in terms of expressive ,images ,or sketches of ,the communicating ,persons. This interface makes ,use of a ,developed ,real time ,emotion ,extraction engine from text. The emotion extraction engine and extraction rules are discussed together with a description of the interface, its limits and future direction of such interface. The extracted emotions are mapped into displayed facial expressions. Such interface can be used ,as a ,platform ,for a number ,of future ,CMC experiments. The developed ,online communication ,interface brings together remotely located collaborating parties in a ,shared electronic space for their communication. In its current state the interface allows the participant to see at a glance,all other online participants and ,all those who ,are engaged ,in communications.,An important ,aspect of the ,interface is that for two users engaged in communication, the interface locally extracts emotional states from the content of typed ,textual sentences automatically. Subsequently it displays discrete expressions mapped,from extracted emotions to the remote screen of the other person. It also analyses/extracts the ,intensity/duration of ,the emotional
Article
We examined the effects of mood and the content (a priori valence and involvement) and formal (presentation modality: text vs. video) characteristics of messages presented on a small screen on emotional responses and involvement among 47 young adults. Mood was induced by autobiographical memories varying in affective valence and arousal. Facial electromyography (EMG) and cardiac interbeat intervals were used as physiological indexes of valence and arousal. Both mood and the emotional tone of a message exerted an independent influence on the emotional response to the message. A strong valence-related mood-congruency effect emerged in predicting involvement. The text modality elicited higher involvement, arousal ratings, and orbicularis oculi EMG activity compared to the video modality when in a depressed mood, whereas the reverse was true when in a joyful, relaxed, or fearful mood. The results point to the possibility of mood-adapted media services.
Article
Despite the increasing use of psychophysiological measures in various research areas, there is a relative paucity of studies on communication, media, and media interfaces that have taken advantage of this approach. This article provides an overview of the use of psychophysiological measures of attention and emotion in media research with the focus on 3 most commonly used measures: heart rate, facial electromyography, and electrodermal activity. Selected media studies that have used psychophysiological methods to test theory-based predictions regarding the role of attentional and emotional factors in message processing are critically reviewed. The article also highlights some methodological and other issues critical for the successful application of psychophysiological methods to problems in media research. In particular, respiratory sinus arrhythmia (RSA), a selective index of parasympathetic nervous system activity, is introduced as a measure that holds particular promise for media research, given that RSA is highly sensitive to changes in attention.
Article
In recent studies of the structure of affect, positive and negative affect have consistently emerged as two dominant and relatively independent dimensions. A number of mood scales have been created to measure these factors; however, many existing measures are inadequate, showing low reliability or poor convergent or discriminant validity. To fill the need for reliable and valid Positive Affect and Negative Affect scales that are also brief and easy to administer, we developed two 10-item mood scales that comprise the Positive and Negative Affect Schedule (PANAS). The scales are shown to be highly internally consistent, largely uncorrelated, and stable at appropriate levels over a 2-month time period. Normative data and factorial and external evidence of convergent and discriminant validity for the scales are also presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Article
discuss several promises as well as potential problems with the circumplex model of emotion / while this model promises to organize much of what we know about emotion, it is nevertheless open to misinterpretation / before detailing these particular strengths and weaknesses, we begin by describing how a circumplex model is applied in the emotion domain / by advocating the circumplex model, a claim is made that the majority of emotional experience can be captured by two affect dimensions [positive affect and negative affect] despite the promise a circumplex model holds for aiding our understanding of emotion, a number of problems need to be understood / one set of problems relates to specific interpretational issues concerning the emotion circumplex: are there basic dimensions in the circumplex and how should the dimensions be named / the second set of problems is broader: what does the circumplex fail to do in describing and explaining the relationships between emotions, and what are the shortcomings of the extant data / we will consider first the interpretational issues and, after that, the broader issues (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
The startle reflex, facial electromyogram (EMG), and autonomic nervous system responses were examined during imagery varying in affective valence and arousal. Subjects (N= 48) imagined affective situations during tone-cued 8-strials. Startle blink magnitudes were larger and latencies faster during negatively valent than during positively valent conditions and during high-arousal than during low-arousal conditions. Greatest heart rate acceleration and fastest and largest skin conductance responses to startle probes occurred during high-arousal imagery. Zygomatic and orbicularis oculi facial muscle activities were higher during high-arousal imagery, whereas corrugator muscle activity was higher during low-arousal imagery. Zygomatic and corrugator activity also varied with emotional valence. The startle and facial EMG responses are most parsimoniously organized by the negative affect (NA) and positive affect (PA) dimensions, respectively. This NA/PA framework integrates previous research, dimensional theories of emotional behavior, and physiological assessment of pathological emotion.