DataPDF Available

Interest as a knowledge emotion: Psychophysiological Classification in the Context of Cultural Heritage

Authors:

Abstract

This paper presents the psychological construct interest as a knowledge emotion and illustrates the importance of interest in a cultural heritage context. Contemporary heritage institutions model installations and artifacts around a passive receivership where content is consumed but not influenced by visitors. We present work towards an adaptive interface that uses psychophysiological measures, physiological computing and a machine learning algorithm, designed to respond to the level of interest of the visitor, in order to deliver a personalised experience within cultural heritage institutions. Two studies are reported which take a subject-dependent experimental approach to record and classify psychophysiological signals using mobile physiological sensors and a machine learning algorithm. The results show that it is possible to reliably infer a state of interest from cultural heritage material, informing future work for the development of a real-time physiological computing system for use within an adaptive cultural heritage experience.
Interest as a knowledge emotion:
Psychophysiological Classification in the Context of Cultural Heritage
Alexander J. Karran
School of Natural Science and Psychology
Liverpool John Moores University
Liverpool, England
a.j.karran@ljmu.ac.uk
Stephen H. Fairclough
School of Natural Sciences and Psychology
Liverpool John Moores University
S.Fairclough@ljmu.ac.uk
Kiel Gilleade
School of Natural Sciences and Psychology
Liverpool John Moores University
gilleade@gmail.com
Abstract This paper presents the psychological construct
interest as a knowledge emotion and illustrates the importance of
interest in a cultural heritage context. Contemporary heritage
institutions model installations and artifacts around a passive
receivership where content is consumed but not influenced by
visitors. We present work towards an adaptive interface that uses
psychophysiological measures, physiological computing and a
machine learning algorithm, designed to respond to the level of
interest of the visitor, in order to deliver a personalised
experience within cultural heritage institutions. Two studies are
reported which take a subject-dependent experimental approach
to record and classify psychophysiological signals using mobile
physiological sensors and a machine learning algorithm. The
results show that it is possible to reliably infer a state of interest
from cultural heritage material, informing future work for the
development of a real-time physiological computing system for
use within an adaptive cultural heritage experience.
Keywords Physiological Computing; Affective Computing;
Machine learning; Cultural Heritage; Adaptive Systems; Human
Computer Interaction
I. INTRODUCTION
The introduction of digital technology has the capability of
increasing the amount of information provided to visitors to a
museum or art gallery [1]. The challenge for technology
design is to enhance the experience of cultural heritage (CH).
In order to understand how the experience of the visitor
may be augmented by technology, we must consider the nature
and quality of CH experience. Previous research described CH
experience in terms of satisfaction, which in turn is determined
by positive expectations of the visitor being fulfilled [2]. The
optimal CH experience has been defined in conceptual terms as
a “total experience” that incorporates aspects of leisure, culture
and social interaction [2, 3]. There are several routes to the
creation of memorable experiences. The visitor may supply a
cognitive and emotional resonance by actively encoding the
visit with their own personal meanings.
A. The cultural heritage experience
The analysis of cultural heritage experience described by
Pine and Gilmore [3] provided a deeper level of analysis by
describing four crucial drivers of visitor experience:
1. entertainment (leisure, narrative)
2. educational (knowledge transfer)
3. aesthetics (pleasure)
4. escapist (immersion)
The first factor refers to capacity of cultural heritage
artifacts to engage the visitor in a cognitive and affective
manner. The educational component of the CH experience
represents the process of knowledge transfer by which the
visitor is informed about artifacts. The aesthetic aspect of
cultural heritage is perhaps the most difficult to understand
because cultural artifacts are capable of evoking a range of
aesthetic responses. Previous definitions of aesthetic
experience have emphasised both information processing and
emotional responses [4], i.e. a cognitive perceptual process
accompanied by a dynamic affective state.
The final factor (escapist) is associated with the degree to
which the visitor is immersed within a mixed reality (i.e. past
present, new technology ancient artifact). The concept of
immersion is often associated with a sense of presence in a
three-dimensional virtual reality (VR) [5]; however, the same
concept may be applied to mixed reality systems such as
augmented reality. The degree of immersion may be
characterised within three levels: (1) engagement (lack of
awareness of time), (2) engrossment (lack of awareness of the
real world), and (3) total immersion (sense of being within a
computerised environment) [6]. Immersion has clear
implications for creating memorable experiences in CH
contexts, particularly using technology to engage and engross
the visitor in a particular artifact.
B. Physiological computing systems
This paper is concerned with technology that is designed to
improve the CH experience of the visitor via the adaptive
provision of information. Physiological computing systems
monitor the physiology of the user and use these data as input
to a computing system [7]. The passive monitoring of
spontaneous changes in physiology indicative of cognition,
emotion or motivation is used to adapt software in real time.
These systems are constructed around a biocybernetic loop [8]
that handles the translation of raw physiological data into
control input at the interface. Passive monitoring of user
psychophysiology can be used to inform intelligent adaptation,
allowing software to respond to the context of the user state in
a personalised fashion.
A physiological computing system could be created to
monitor the CH experience in real-time by quantifying the state
of the visitor and using these data to personalize the provision
of information via a process of “adaptive curation”. To perform
this act of personalization, the physiological computing system
must be sensitive to those psychological dimensions
underpinning the four facets of the CH experience. It is
proposed that activation, cognition and valence serve an
interactive role in the CH experience with cognitive stimulation
playing a primary role in the educational aspect and activation
and valence capturing the emotional and aesthetic aspect of
visitor experience. Both cognitive stimulation and emotional
engagement interact in order to yield the escapist or immersive
facet of the experience.
II. BACKGROUND
The psychological conceptualisation of affective experience
falls into two distinct theoretical camps. Theories of basic
emotions, e.g. happiness and fear, argue that emotional
experience may be divided into discrete and independent
categories [9]. This model contrasts with the circumplex
model developed by Russell [10, 5] that represents emotional
experience within a two-dimensional space consisting of
arousal/activation (alert - tired) and valence (happy - sad).
Unlike the basic emotions theory, the circumplex model
emphasises the association between different categories of
emotional experience via the common dimensions of activation
and valence. It is assumed that cultural artifacts that are
stimulating, both in a cognitive and an emotional sense, will
increase the activation level of the visitor and responses will
span the range of positive or negative affect.
The experience of a cultural heritage environment,
regardless of whether it is a museum or gallery, is shaped by
exploratory behaviour driven by the interest and curiosity of
the visitor. A physiological computing system must build on
these psychological motivations in order to capture the
experience of the visitor. The concept of interest as a
psychological entity was described by Berlyne [11] in terms of
increased arousal and sensation-seeking, i.e. objects inspire
curiosity via novelty and emotional conflict. This concept was
expanded by Silvia [12, 13] to incorporate a cognitive
dimension, i.e. interest driven by stimulus complexity. Both
cognitive and emotional facets of interest were explored by
Hidi and Renninger [14] who referred to the former as
perceptual/representational processes, which was accompanied
by a sense of positive emotion derived from intellectual
engagement; it was argued that positive emotion occurred even
during engagement with negative material.
A. Conceptual model of interest
A conceptual model of interest was developed based upon a
review of the literature. This conceptual model consisted of six
sub-components of perceptual representational processes, three
of which are cognitive in nature and three emotional factors.
The cognitive sub-components consisted of:
Novelty, i.e. whether the object or exhibit was familiar or
unexpected/unconventional/different
Comprehension, i.e. whether the representation/function of
the object was clearly understood
Complexity, i.e. whether the perceptual complexity of the
object is high or low
The emotional components of interest owe much to the
work of Berlyne [11] and are described as follows:
Activation/Arousal, i.e. whether consideration of the object
was stimulating or not
Attraction, i.e. whether object was viewed as either
attractive or repellent
Valence, i.e. whether viewing the object made the person
feel happy or sad
This model of interest was derived to represent the visitor
experience in a cultural heritage setting. The model includes
both cognitive and emotional engagement believed to underpin
memorable experiences in museums and galleries. These
concepts were explored via an online survey conducted where
participants viewed sixty art images (N=1023) and provided
ratings on all six dimensions for each. This survey revealed a
high degree of correlation between Novelty, Complexity and
Comprehension, i.e. novel objects were hard to understand. It
was also found that Attraction was positively related to
Valence, i.e. attractive objects made people happy. Therefore,
we reduced our conceptual model of interest to a single
cognitive component (representing a high/low cognitive
response to the complexity/ease of comprehension associated
with the stimulus), activation (high/low stimulation) and
valence (positive/negative affect).
B. Operationalising the model
Operationalising these psychological variables as
psychophysiological measures is a key step in the development
of a CH physiological computing system. The cognitive
component of interest was identified with activation of the
rostral prefrontal cortex, which has been linked to working
memory and attentional control [15]. This variable was
captured using spontaneous EEG measures of electrocortical
activation, it is known that alpha activation has a converse
relationship with brain activation [16], i.e. higher alpha activity
is associated with reduced brain activation. The activation
component was captured via the level of skin conductance
(SCL) and supplemented by measuring heart rate (HR); SCL is
highly sensitive to sympathetic activity [17] and HR captures
both sympathetic and parasympathetic components of the
autonomic nervous system. Valence, generally measured in
psychophysiology using facial EMG [18], was deemed too
intrusive for our needs, hence we captured valence by
measuring EEG frontal asymmetry. It has been hypothesized
that greater left activation of the prefrontal cortex is associated
with positive affect whereas greater right side activation is
linked to negative affect [19].
This array of psychophysiological measures is designed to
deliver a multidimensional representation of interest to be used
to quantify the interest level of an individual in a dynamic
fashion, in order to inform a process of adaptive curation in
near-real time. To test this concept, two experimental studies
were designed to record and classify psychophysiological
responses to CH material. Our approach combines the interest
model with psychophysiological data and a machine learning
algorithm in order to distinguish between stimuli that are high
or low with respect to the level of interest provoked in the
viewer.
III. EXPERIMENTAL STUDIES: ELICITATION OF INTEREST
USING CULTURAL HERITAGE MATERIAL
The first study set out to create a virtual heritage
installation that replicated in part, a late 18th century Valencia
kitchen mosaic (installed at the Museo Nacional de Artes
Decorativas in Madrid). The second study was undertaken in
situ at the Foundation for Art and Creative Technology
(FACT) in Liverpool. The former allowed participants in the
study to stand in a natural fashion, while simultaneously
viewing the mosaic and listening to audio narrative about
elements of the representation. The latter study was designed to
present participants with audio and video content associated
with a CH artifact. The studies were designed with the
following goals:
To measure and classify psychophysiological reactivity in
response to CH content presented as visual and audio
stimuli
To define the psychophysiological variance as a two
condition level of interest (high and low) consisting of
three dimensions: activation, cognition and valence
To determine the optimum method of gathering subjective
response data and observe its effect on classifier
performance
To evaluate the performance of the Support Vector
Machine (SVM) classification algorithm[20, 21] for real-
time application and the precision of the classifier, when
compared to subjective response data
To evaluate the effect of differing feature sampling rates on
classifier performance
A. Experimental Tasks
For study one, participants were asked to view a visual
representation of the mosaic and listen to an audio narration
salient to the highlighted parts of the representation, then to
answer post-hoc questions relating to interest level per auditory
segment.
For study two, Participants were asked to view a heritage
presentation, consisting of a continuous audio-visual media
partitioned into a number of “themed” content “blocks”. Three
themes were presented each pertaining to the work of three
specific film-makers and artists. The content blocks within
each theme were delineated by pre-determined categories: the,
Context, Work and Style of the artist under consideration.
After each theme, participants were asked to complete a
questionnaire which coincided with the stimulus presentation,
such that subjective judgments were given for context, work
and style.
B. Methods
10 participants 2 male 8 female, aged 19-75 took part in the
first study, physiological responses from the autonomic system
were measured during experimental sessions, using the
Electrocardiogram (ECG, sampled from the torso) and SCL
(second and forth finger, non-dominant hand) channels of the
Mind Media Nexus X Mk II (sampled at 512Hz). Four
channels of electroencephalographic (EEG) data were recorded
using the Enobio wireless 4-channel sensor (sampled at 250Hz)
with ground contacts on left ear lobe and inner ear. A Biosemi
EEG cap was fitted and aligned to ensure sensor placement,
electro-conductive gel was added to sites fp1, fp2, f3 and f4
[22] and electrodes attached.
For the second study, 8 participants 5 male 3 female, aged
20-40 took part, psychophysiological response data was
collected in a similar way to study one with the notable
exception of EEG data. As this was an in-situ study, it was
necessary to dispense with the Biosemi EEG cap to allow for
the ergonomic considerations of participants and speed of
fitting. To this end, three channels of EEG data were recorded
using the Enobio EEG sensor with mobile headband fitted and
three dry electrodes placed at sites fp1, fp2 and fpz.
C. Procedures
For study one, instruction about the experimental procedure
was given and participants were asked to complete a consent
form in accordance with the Liverpool John Moores Ethical
Committee, then fitted with a mobile pouch to hold the nexus
sensor hardware at the hip. Electrodes for ECG and SCL were
placed on the torso and fingers, the Biosemi sensor cap was
fitted and electrodes attached. Participants were asked to stand
in a relaxed position approximately 2 meters in front of a 2*3
meter projection screen. This was followed by the audio-visual
presentation of the Valencia kitchen. The presentation of the
kitchen stimulus was linear and timed to progress through the
narrative, giving four stories (average.15s) consisting of 3
factual elements. When the presentation was completed each
participant was asked to rate which two stories were perceived
to be the most interesting out of the four that were presented.
For study two, after instruction, participants were asked to
complete a consent form, and then fitted with the Nexus sensor
technology for ECG and SCL. The Enobio headset was then
fitted for comfort and the dry sensors aligned for signal quality.
Participants were asked to sit in a relaxed position
approximately half a meter in front of a large computer screen
and then asked to view a calming video for 10 minutes to allow
a psychophysiological baseline to be established, following
which counterbalanced stimulus content was presented.
To determine the memorability of the material and provide
class labels for the psychophysiological response data,
participants were asked to complete a questionnaire consisting
of three Likert scales ranked 1 10. These aligned to the 3
dimensions of the interest model; Activation: tired passive 0 to
activated alert 10; Valence: sad angry 0 to happy cheerful 10;
Cognition: low 0 to high 10. Participants were offered access to
the content to aid in content recall if needed.
D. Analysis study one
Prior to commencing classification analysis of the
physiological data, features were derived from measures of
heart rate, skin conductance and EEG. This resulted in a total
of 9 features for each of the stimulus events. These features
were further subdivided into components of a three
dimensional interest model, such that each feature set created a
unique classifier feature vector for each element.
Activation : mean heart rate, inter-beat interval (iBi) and
mean skin conductance level
Cognition : Where the ratio is expressed as (power 12-
30Hz) divided by (power 8-12Hz) at fp1, fp2, f3, f4 (1)
Valence : Where the ratio is expressed as lognormal of
(power), subtracting right from left hemispheric activity at
sites (fp1,fp2) and (f3,f4) (2)
  
(1)
  
(2)
This approach has a number of advantages. Each feature
vector is identified as a separate element of the model, feature
sets can be combined as a fusion of features, and the effect of
each feature set or fusion of features on classifier class recall
can be evaluated.
Each participant’s data were analysed separately to
determine the recall accuracy of the SVM classifier for
individual participant responses. The SVM classifier is a
supervised pattern recognition algorithm, requiring an n
dimensional vector (observation) and an associated label (class)
for training. This training set is then used as the basis for
classifying new instances of data into its respective class. For
this study the classification analysis was performed using the
SVM implementation by prtools [23] within Matlab 2011Rb.
Each feature set was tested using 5-fold (k-fold) cross-
validation. In k-fold cross-validation, k-1 folds are used for
training and the last fold is used for evaluation. This process is
repeated k times, leaving one different fold for evaluation each
time. Furthermore, in order to test the capacity of the classifier
to generalise across all participants, the feature data was
combined into one dataset and classified twice: firstly, using 5
fold cross-validation and secondly, by splitting the dataset in
two parts selected at random, one for training and one for
testing.
E. Analysis study two
As above, features were derived from measures of heart
rate, skin conductance and EEG, resulting in a total of 10
features. The baseline period and themed content from block
one are used to “prime” the participant’s psychophysiology
towards the content presented in blocks 2 and 3 (an average of
10 stimulus events) and not used within the classifier
evaluation. As mentioned previously, these features were
further subdivided into components of the interest model.
Activation : For heart rate (iBi), mean, maximum and
standard deviation; for skin conductance level, mean,
maximum and standard deviation extracted every 2
seconds for each content stimulus epoch
Cognition : EEG data was derived from a fast fourier
transform of total amplitude spectra using a 2 second
hanning window for each stimulus epoch, where the ratio
is expressed as lognormal (power) divided by
(power) at sites fp1, fp2, fpZ
Valence : Where the ratio is expressed as lognormal of
(power) subtracting right from left hemispheric activity at
sites (fp2,fp1) (2)
As with the first study, a subject dependent approach was
taken to analysing psychophysiological data to determine the
recall accuracy of the SVM classifier. However, the SVM
classifier implemented for this study was part of the
bioinformatics module within Matlab. To evaluate classifier
performance, two cross-validation methods were chosen. The
first method applied a loose grid search algorithm with 5-fold
cross validation to the psychophysiological response data from
content blocks 2 and 3. The loose grid search provided optimal
settings for the box constraint and sigma values of the SVM
radial basis function (RBF) kernel, while 5-fold cross-
validation provided the recall performance of the classifier. The
second method applied the same loose grid search and “hold-
out” cross-validation, this method partitions the data into two
parts, by randomly assigning data to either a training or testing
set, ensuring that the classifier is trained and tested with novel
data and is analogous to a real world task. This method of
cross-validation has been shown to provide a more accurate
assessment of classifier performance when applied to small
datasets, such as those gained from real-time applications [24].
In both cases, the class labels used to train the classifier
were obtained from the subjective data taken during the
experimental task. To derive the binary class label from the
likert scaled scores, normalisation was applied to the subjective
scores in the form of:


 (3)
Where is the sum of subjective scores for each
dimension of the model (activation, cognition and valence),
minC
and
maxC
are the minimum and maximum of the
population of scores for each content block, the result is a
population of normalised scores. To set the threshold for class
assignation, the median of this population was determined,
above the median was labeled high and below low interest.
IV. RESULTS
The results obtained from the virtual heritage installation
study are summarised in table 1, these figures represent the
classifier recall accuracies from the subject-dependent
classification of the feature data. The feature sets (activation,
cognition and valence) were classified alone and in
combination, to determine which permutation of features
provided the best class recall accuracy over all participants.
The table data indicates that the fusion of raw activation and
cognition features afforded a mean class recall accuracy across
all participants of 80%, with a minimum 66% and maximum
100% spread over all participants.
This significant classification rate offers strong evidence,
that the combination of activation and cognition features
affords an effective method, from which to ascertain a user’s
level of interest in a cultural heritage setting. However,
examining the individual recall rates in isolation shows that, for
some participants, this combination of features resulted in
lower class recall accuracies, highlighting the influence of
Feature(s) P1 P2 P3 P4 P5 P6 P7 P8 P9 P1 0 Mean Recall
A0.44 0.72 0.80 0.83 0.90 0.80 0.63 0.89 0.67 0.90 76%
A,C 1.00 0.70 0 .66 0.90 0 .77 0.87 0 .60 0.80 0.7 0 1.0 0 80%
A,V 0.66 0.62 0.66 1.00 0.90 0.78 0.45 0.88 0.66 1.00 76%
A ,C,V 0.81 0.62 0.70 0.90 0.80 0.67 0.60 0.80 0.70 1.00 76%
C,V 1.00 0.66 0.66 0.50 0.66 0.77 0.80 0.77 0.81 1.00 76%
C0.87 0.90 0.72 0.55 0.50 0.44 0.80 0.70 0.87 1.00 74%
V0.75 0.55 0.60 0.54 0.83 0.60 0.50 0.87 1.00 0.80 70%
Participant 1 2 3 4 5 6 7 8 Mean Recall
CrossValidation 75.4 71.6 8 8.2 62.7 63 .4 56.0 72 .3 75.7 70 .7
HoldOut 68.4 7 9.0 93 .6 67.2 69 .8 59.0 77.2 8 0.7 74.4
Participant 1 2 3 4 5 6 7 8 Mean Recall
CrossValidation 69.0 72.3 8 6.9 65.3 63 .4 56.7 73 .1 76.4 70 .4
HoldOut 75.0 7 5.8 91 .9 67.2 67 .9 60.7 79.0 8 0.7 74.8
individual differences in physiological responses towards the
heritage material.
Comparing the classifier recall accuracies from other
feature sets, it can be seen that the features of activation alone
are only 3% less accurate overall than those of the combined
activation and cognitive feature sets, with a maximum of 76%
mean recall accuracy. However, although this result was
promising, there can be seen a greater (negative) intra-subject
classification variation in recall accuracies, when compared
with the combined feature set. Combining the features of
activation with valence, or cognition with valence provided no
clear benefit to recall accuracies with a maximum recall
accuracy of 76% respectively for raw feature data.
Findings from the literature [25] and previously completed
work indicate that problems exist with classifier generalisation
accuracy due to magnitude differences in psychophysiological
responses between individuals. In generalisation tests the
classifier reported a steep drop in accuracy in line with these
findings. However, combining all three feature sets into a
single vector resulted in a predictive accuracy of 65%, still
15% above that of chance. These findings formed the basis
from which to build the methodology and analysis framework
for the second in-situ study, informing further development of
the subject-dependent classifier approach.
Turning now to the experimental results from study 2,
Table 2 summarises the classifier recall accuracies for the
fusion of activation and cognition features. Results from the
previous study indicate that this combination of features should
present with the highest classification recall accuracy, this
finding holds true within the current study. Mean class recall
accuracy for both cross-validation methods were respectable
and well above chance, with the holdout method reporting
74.8% (range 59.0 - 93.6%). However, in line with previous
findings there can be seen a more than marginal intra-subject
classification variation in recall accuracies.
Interestingly, when the features of activation, cognition and
valence are combined into a single vector, the intra-subject
classification variation becomes less pronounced and mean
recall accuracies remain largely unaffected, this can be seen in
Table 3, where in some cases individual recall accuracies rose
in response to the addition of valence features with a range of
67.2-91.1% compared to 59-93.6% for the combination of
activation and cognition. Illustrating, that for this study,
participants responded more strongly to the emotional context
and prosody embedded in the source material. This
combination of features, lead to greater variation within the
physiological responses which in turn added a larger degree of
separation within the vector space between the two classes
allowing for more consistent classification.
To test the generalisation capacity of the classifier, the
datasets for all participants were combined into one larger
dataset and tested using the above methods. The results from
this analysis were promising, with the classifier reporting 64.54
and 66.52% for 5-fold and holdout methods respectively using
all features combined. Applying scaling and normalisation to
the feature set resulted in a negative impact on recall
accuracies, revealed as a mean decrease in recall accuracy of
4% for both validation methods. These results are significantly
above that of chance and show the promise of the SVM as a
classifier for use in real-time psychophysiological classification
tasks.
V. DISCUSSION
The results from these studies provide evidence, that the
combination of activation, cognition and valence features,
coupled with a SVM classifier and a subject-dependent
classification approach, can reliably infer the “knowledge
emotion” interest within a cultural heritage context. It is
interesting to note however, that these results also highlight the
possibility that an approach, based upon an ensemble of
classifiers for each dimension of the interest model, may
provide even greater recall accuracies when compared to the
combined approach.
This approach would utilise the binary classification from
each component of the interest model, and apply second order
logic to map them into eight states, representative of an interest
“experience”. Such that, IF activation = High AND cognition =
High AND valence = Positive: INTEREST = high. This eight
state model transposes well onto our current research and can
be pursued in parallel and integrated into a real-time decision
engine. Moreover, the transposition of three binary
classifications into eight states could potentially be used as the
basis for a more comprehensive interaction and adaption
model, allowing CH institutions to gather more detailed visitor
“interest” statistics about installations and associated content
and to create more memorable heritage experiences.
We set an initial accuracy floor of 72% below which
interactive systems using this approach within a bio-sensing
component would become unusable. This figure was arrived at
arbitrarily, thus requiring further research to determine the
optimum balance between synthetic classification accuracy and
quality of user experience. The next task is to integrate the bio-
sensing component into an interactive system and evaluate its
performance using receiver operator characteristic techniques
and real-time user feedback. This will aid in the determination
of what level of accuracy is acceptable for users of the system,
and inform the creation of an adaptivity model for interactive
CH applications. This is currently work in progress and a real-
time interactive heritage application is in development. We
TABLE 1 ALL FEATURES MEAN RECALL
TABLE 2 FUSION OF ACTIVATION AND COGNITION
envision many possible applications of this approach within the
context of cultural heritage, such as automated or semi-
automated recommendation of cultural heritage content
informed by real-time psychophysiological assessment (a
digital curator) or “interest” profiling involving implicit
tagging of heritage material to build up heat maps that use
interest as a basis to inform future interactions and build
cultural heritage installations that imbue artifacts with a sense
of modernity, whilst at the same time preserving any cultural
and historical significance.
VI. CONCLUSION
The overarching goal of this research was to answer the
questions Can we use physiological computing for adaptive
information provision in a CH context?”, Will a sustained
state of interest using personalised information provision
enhance the CH experience? we tentatively posit an answer of
yes. The results from the two studies presented here show that
it is possible to reliably infer a state of interest from
psychophysiological signals. However, in order for an
interactive CH physiological computing system to be fully
realised, the approach we have discussed must be applied
outside of the laboratory and restricted simulated environments
and tested in the field. To this end a real-time interest state
classification and information adaption system is under
development. The success of this system will be contingent on
overcoming a number of technical issues arising from
naturalistic visitor behaviors, such as detecting artifacts within
physiological signal acquisition caused by movement patterns
(walking, acceleration and hand gestures). However, we are
confident that the technical difficulties are a matter of iterative
design and testing and not overwhelming.
ACKNOWLEDGEMENTS
This research was funded by the CEC under the
ARtSENSE project (http://www.artsense.eu). We would like
to thank Ana Cabrera and her colleagues at MNAD for
supplying English text for the first study. We would also like
to acknowledge the work of Roger McKinley and Clara Casian
from FACT who developed the material used in study two.
REFERENCES
[1] M. Ott, & F. Pozzi, Towards a new era for Cultural Heritage
Education: Discussing the role of ICT. Computers in Human
Behaviour, 27, 1365-1371, 2011.
[2] C. De Rojas, & C. Camarero, “Visitors’ experience, mood and
satisfaction in a heritage context. Evidence from an interpretation
center. Tourism Management. 29, 525-537, 2008.
[3] J, Pine. & J,H, Gilmore, Welcome to the experience economy.
Harvard Business Review, 97-107, 1998.
[4] H. Leder, B. Belke, A. Oesberst, & D. Augustin, A model of aesthetic
appreciation and aesthetic judgement. British Journal of Psychology,
95, 489-508, 2004.
[5] J.A. Russell,. Core affect and the psychological construction of
emotion. Psychological Review, 110, 145-172, 2003.
[6] C. Jennett, A. L. Cox, P. Cairns, S. Dhoparee, A. Epps, T. Tijs & A.
Walton, Measuring and defining the experience of immersion in
games. International Journal of Human-Computer Studies, 66, 641-661,
2008..
[7] S. H. Fairclough, Fundamentals of Physiological Computing.
Interacting With Computers, 133-145, 2009.
[8] S. H. Fairclough, and K. Gilleade, Construction of the biocybernetic
loop: a case study. In Proceedings of the 14th ACM International
Conference on Multimodal Interaction (Santa Monica, 2012). ACM.
[9] P. Ekman, An argument for basic emotions. Cognition and Emotion,
6,169-200, 1992.
[10] J. A. Russell, A circumplex model of affect. Journal of Personality
and Social Psychology, 39, 1161-1178, 1980.
[11] D. E. Berlyne, Conflict, Arousal, and Curiosity. London: McGraw-
Hill Book Company, 1960.
[12] P. J. Silvia, InterestThe Curious Emotion. Current Directions in
Psychological Science, 17(1), 57-60,2008.
[13] P. J. Silvia, Confusion and interest: The role of knowledge emotions in
aesthetic experience. Psychology of Aesthetics, Creativity, and the
Arts, 4(2), 75-80, 2010.
[14] S. Hidi & K. A. Renninger, The four-phase model of interest
development. Educational Psychologist, 41(2), 111127, 2006.
[15] N. Ramnani, & A. M. Owen, Anterior prefrontal cortex: insights into
function from anatomy and neuroimaging. Nat Rev Neurosci. 5, 3, p.
184-94, 2004.
[16] R. I. Goldman, J.M. Stern, J. Engel & M.S. Cohen, Simultaneous EEG
and fMRI of the alpha rhythm. Neuroreport 13, 2487-2492, 2002.
[17] W. Boucsein, Electrodermal Activity. Plenum Press, New York, 1992.
[18] J. T. Cacioppo, L.K. Bush, L.G. Tassinary, Microexpressive facial
actions as a function of affective stimuli: Replication and extension.
Personality and Social Psychology Bulletin 18, 515-526,1990.
[19] R. J. Davidson, What does the prefrontal cortex "do" in affect:
perspectives on frontal EEG asymmetry research. Biological
Psychology 67, 219-233, 2004.
[20] C. J. C. Burges. “A tutorial on support vector machines for pattern
recognition.” Knowledge Discovery and Data Mining, 2, 1998.
[21] J. C. Platt. “Fast training of support vector machines using sequential
minimal optimization.” In Advances in kernel methods, Bernhard
Schlkopf, Christopher J. C. Burges, and Alexander J. Smola (Eds.). MIT
Press, Cambridge, MA, USA 185-208, 1999.
[22] H. H. Jasper, The 1020 electrode system of the International
Federation. Electroenceph. clin. Neurophysiol.10, p370375, 1958.
[23] R.P.W. Duin, P. Juszczak, P. Paclik, E. Pekalska, D. de Ridder, D.M.J.
Tax, S. Verzakov “PRTools4.1, A Matlab Toolbox for Pattern
Recognition”, Delft University of Technology, 2007
[24] A. Isaksson, M. Wallman, H. Göransson & M.G. Gustafsson, Cross-
validation and bootstrapping are unreliable in small sample
classification”. Pattern Recognition Letters, Volume 29, Issue 14, Pages
1960-1965, 2008
[25] D. Novak, M. Mihelj, M. Munih, A survey of methods for data fusion
and system adaptation using autonomic nervous system responses in
physiological computing”. In teracting with Computers, Volume 24,
Issue 3, Pages 154-172. 2012
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Physiological computing represents a mode of human–computer interaction where the computer monitors, analyzes and responds to the user’s psychophysiological activity in real-time. Within the field, autonomic nervous system responses have been studied extensively since they can be measured quickly and unobtrusively. However, despite a vast body of literature available on the subject, there is still no universally accepted set of rules that would translate physiological data to psychological states. This paper surveys the work performed on data fusion and system adaptation using autonomic nervous system responses in psychophysiology and physiological computing during the last ten years. First, five prerequisites for data fusion are examined: psychological model selection, training set preparation, feature extraction, normalization and dimension reduction. Then, different methods for either classification or estimation of psychological states from the extracted features are presented and compared. Finally, implementations of system adaptation are reviewed: changing the system that the user is interacting with in response to cognitive or affective information inferred from autonomic nervous system responses. The paper is aimed primarily at psychologists and computer scientists who have already recorded autonomic nervous system responses and now need to create algorithms to determine the subject’s psychological state.
Article
Full-text available
The effects of communicative intent and stimulus affectivity on facial electromyogrqphic (EMG) activity were investigated. Subjects viewed slides of pleasant, neutral, or unpleasant social or nature scenes under no instruction, inhibit-expression instructions, and amplify-expression instructions. Results revealed that facial EMG activity was highest in the amplify and lowest in the inhibit condition; EMG activity over the corrugator supercilii region varied as a function of the affective valence of the stimuli regardless of instructional condition; and facial EMG activity did not differ when subjects were exposed to slides of nature versus social scenes that were matched for rated pleasantness. These results suggest that facial efference can be altered by both affective and communicative processes even when it is too subtle to produce a socially perceptible facial expression.
Article
Full-text available
Factor-analytic evidence has led most psychologists to describe affect as a set of dimensions, such as displeasure, distress, depression, excitement, and so on, with each dimension varying independently of the others. However, there is other evidence that rather than being independent, these affective dimensions are interrelated in a highly systematic fashion. The evidence suggests that these interrelationships can be represented by a spatial model in which affective concepts fall in a circle in the following order: pleasure (0), excitement (45), arousal (90), distress (135), displeasure (180), depression (225), sleepiness (270), and relaxation (315). This model was offered both as a way psychologists can represent the structure of affective experience, as assessed through self-report, and as a representation of the cognitive structure that laymen utilize in conceptualizing affect. Supportive evidence was obtained by scaling 28 emotion-denoting adjectives in 4 different ways: R. T. Ross's (1938) technique for a circular ordering of variables, a multidimensional scaling procedure based on perceived similarity among the terms, a unidimensional scaling on hypothesized pleasure–displeasure and degree-of-arousal dimensions, and a principal-components analysis of 343 Ss' self-reports of their current affective states. (70 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
An abstract is not available.
Article
The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and non-separable data, working through a non-trivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light.
Article
Building on and extending existing research, this article proposes a 4-phase model of interest development. The model describes 4 phases in the development and deepening of learner interest: triggered situational interest, maintained situational interest, emerging (less-developed) individual interest, and well-developed individual interest. Affective as well as cognitive factors are considered. Educational implications of the proposed model are identified.
Conference Paper
The biocybernetic loop describes the data processing protocol at the heart of all physiological computing systems. The loop also encompasses the goals of the system design with respect to the anticipated impact of the adaptation on user behaviour. There are numerous challenges facing the designer of a biocybernetic loop in terms of measurement, data processing and adaptive design. These challenges are multidisciplinary in nature spanning psychology and computer science. This paper is concerned with the design process of the biocybernetic loop. A number of criteria for an effective loop are described followed by a six-stage design cycle. The challenges faced by the designer at each stage of the design process are exemplified with reference to a case study where EEG data were used to adapt a computer game.
Article
What makes something confusing? Confusion is a common response to challenging, abstract, and complex works, but it has received little attention in psychology. On the basis of appraisal theories of emotion, I suggest that confusion and interest have different positions in a 2-dimensional appraisal space: Interesting things stem from appraisals of high novelty and high comprehensibility, and confusing things stem from appraisals of high novelty and low comprehensibility. Two studies—a multilevel correlational study and an experiment that manipulated comprehensibility—found support for this appraisal model. Confusion and interest are thus close relatives in the family of knowledge emotions. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Despite their interest in why people do what they do, psychologists typically overlook interest itself as a facet of human motivation and emotion. In recent years, however, researchers from diverse areas of psychology have turned their attention to the role of interest in learning, motivation, and development. This article reviews the emerging body of work on the psychology of interest, with an emphasis on what contemporary emotion research has learned about the subject. After considering four central questions—Is interest like other emotions? What functions does interest serve? What makes something interesting? Is interest merely another label for happiness?—the article considers unanswered questions and fruitful applications. Given interest's central role in cultivating knowledge and expertise, psychologists should apply research on interest to practical problems of learning, education, and motivation.