Conference PaperPDF Available

Monitoring and Supporting Engagement in Skilled Tasks: From Creative Musical Activity to Psychological Wellbeing


Abstract and Figures

While good physical health receives more attention, psychological wellbeing is an essential component of a happy existence. An everyday source of psychological wellbeing is the voluntary practice of skilled activities one is good at. Taking musical creation as one such skilled activity, in this work we employ an interaction method to monitor varying levels of engagement of musicians improvising on a desk-top robotic musical interface (a network of intelligent sonic agents). The system observes the performer and estimates her/his changing level of engagement during the performance , while learning the musical discourse. When engagement levels drop, the musical instrument makes subtle interventions, coherent with the compositional process, until the performer's engagement levels recover. In a user study, we observed and measured the behaviour of our system as it deals with losses of performer focus provoked by the controlled introduction of external distractors. We also observed that being engaged in our musical creative activity contributed positively to participants' psychological wellbeing. This approach can be extended to other human activities.
Content may be subject to copyright.
Monitoring and Supporting Engagement in Skilled Tasks: From Creative
Musical Activity to Psychological Wellbeing
Juan Carlos Vasquez, Koray Tahiro˘
glu, Niklas P¨
Department of Media
Aalto University
School of ARTS
FI-00076 AALTO Finland
Johan Kildal
naki Goenaga 5
Eibar 20600
Teemu Ahmaniemi
Nokia Technologies
Espoo, Finland
While good physical health receives more attention, psy-
chological wellbeing is an essential component of a happy
existence. An everyday source of psychological wellbeing
is the voluntary practice of skilled activities one is good at.
Taking musical creation as one such skilled activity, in this
work we employ an interaction method to monitor varying
levels of engagement of musicians improvising on a desk-
top robotic musical interface (a network of intelligent sonic
agents). The system observes the performer and estimates
her/his changing level of engagement during the perfor-
mance, while learning the musical discourse. When en-
gagement levels drop, the musical instrument makes sub-
tle interventions, coherent with the compositional process,
until the performer’s engagement levels recover. In a user
study, we observed and measured the behaviour of our sys-
tem as it deals with losses of performer focus provoked by
the controlled introduction of external distractors. We also
observed that being engaged in our musical creative ac-
tivity contributed positively to participants’ psychological
wellbeing. This approach can be extended to other human
Through life, individuals develop skills in activities that
they find interesting and at which they excel. Such skilled
activities often combine intellectual, crafting and creative
dimensions: from cooking and knitting to writing, drawing
or, in this case, performing with a new musical interface.
Practising such activities in an autonomous and voluntary
way can bring great satisfaction and sense of purpose to a
person, who finds the opportunity to challenge the limits of
one’s own acquired mastery. The confluence of these three
characteristics in an occupation - autonomy, mastery and
purpose - has been identified and persuasively proposed as
a popular recipe for personal fulfilment at work [1]. In the
wider scope of life, these three characteristics are also part
Copyright: c
2017 Juan Carlos Vasquez, Koray Tahiro ˘
glu, Niklas P ¨
onen et
al. This is an open-access article distributed under the terms of the
Creative Commons Attribution 3.0 Unported License, which permits unrestricted
use, distribution, and reproduction in any medium, provided the original author
and source are credited.
of a set of six ingredients that have been identified as im-
portant for a healthy, actively-positive psychology in adult
life: autonomy, (environmental) mastery, purpose (in life),
self acceptance, positive relations with other people and
personal growth [2]. Psychological wellbeing is an essen-
tial aspect of general wellbeing that traditional medicine
does not take care of as a priority in every case. However,
it is essential for a happy existence, and being engaged in
skilled tasks provides much of such satisfaction and subse-
quent psychological wellbeing [3].
In the work that we present here, we focus on exposing
the centrality of the individual’s engaging experiences in
psychological-wellbeing enhancing musical activities. We
seek to amplify the experience by mediating between the
person and the activity in two complementary ways: (i) by
monitoring the level of engagement of the person during
a musical activity, and (ii) by having the object of the ac-
tivity itself respond in subtle but meaningful ways to the
music composition led by the person, in order to main-
tain consistently high levels of engagement. We build on
work that we developed earlier [4], regarding the machine
recognition and monitoring of the engagement level of an
individual that is immersed in a musical task with digi-
tal environments. The work so far has focused on recog-
nising levels of immersion of musical performers during
creative improvisation with a desktop Network Of Intelli-
gent Sonic Agents (NOISA), the musical instrument itself.
By combining the monitoring of facial expressions, bodily
movements, actions of the hands and the evolution of the
sonic production, an initial setup has been validated to suc-
cessfully estimate levels of engagement at each time of the
creative musical process [5].
As the main technical contribution of this paper, we add
to the engagement monitoring functionality that NOISA
had, a new response module, which preserves high levels
of engagement in a musical activity when these appear to
fade. In this paper, we report a new study conducted in
the new NOISA environment that aimed at evaluating the
functionality of the response module and the music compo-
sition task’s effect on the participant’s psychological well-
being. First, we evaluated the system’s functionality to
identify instances in which we observe a drop in the level
of engagement of the person with the musical activity. Sec-
ondly, we evaluated the response module’s functionality to
intervene in subtle ways in order to “rescue” dropping lev-
Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland
els of engagement back to higher levels that are associated
with a satisfactory creative experience. Lastly, as happi-
ness and wellbeing crucially depend on the activities we
engage in [3], in this user study we aimed at understand-
ing whether a satisfactory musical experience could indeed
contribute towards the preservation of psychological well-
being, as we proposed in the beginning.
In spite of being a commonly mentioned parameter in mu-
sic research, the mere act of obtaining a unified and con-
sistent definition for ‘engagement’ represents a surprising
challenge. In the field of education, Appleton [6] points
the discussion towards separating engagement from mo-
tivation: the latter [7] provides an answer to the reason-
ing behind a specific behaviour depending on the charac-
teristics of the personal energy invested in said situation,
whereas engagement refers to the level of focused involve-
ment [8], or “energy in action, the connection between per-
son and activity” [9]. When applied to musical activity, en-
gagement may also have a positive connotation, described
as a state of sustained, undivided attention and emotional
connection towards creative musical performance [10] or
sometimes as a neutral parameter prone to be measured
and evaluated in interdependent values for attention, emo-
tion and action [11].
Similarly, there are other studies exploring music as a
creative activity that contributes to the general wellbeing.
An example demonstrates how active music-making im-
proves wellbeing and overall comfort in a large sample of
hospitalised children with cancer [12]. This study draws
upon previous research featuring live music therapy that
promotes interaction through voice and body expressions,
recognising the benefits of interactivity in opposition of lis-
tening passively to music as a therapy [13]. While previous
studies delineate a methodology for therapy rather than the
design of an interactive system, there are examples of in-
teractive tools aimed at positively impacting in wellbeing
through rhythmical movement with music as a central ele-
ment [14, 15]. There is also the precedent of an interactive
application to engage patients with dementia in active mu-
sic making without pre-existing skills [16]. In the same
vein, we found an interactive technology tested in children
with disabilities and aimed to study how music interaction
can “become potentially health promoting” [17].
As a concept, psychological wellbeing is subjective, rel-
ative and prone to change with ease. However, it is pos-
sible to find specific sonic characteristics that have proven
to affect positively an individual’s perception of comfort
and wellbeing. For instance, in the context of music, some
studies indicate that fast and slow tempos are associated
with happiness and sadness, respectively, as are major and
minor modes [18, 19]. Loud music has proven to be in
cross-cultural studies as an universal cue for anger [20].
Timbre is ambivalent, although several studies indicate that
attenuation of high frequencies can be a difference between
anger (unfiltered) and tenderness (attenuation) [19]. Al-
though affective associations of both tempo and mode are
fairly well established, effects of other musical character-
istics are poorly understood. On the cognitive level, David
Huron’s ITPRA (Imagination-Tension-Prediction-Response-
Appraisal) theory studies the relationships between affec-
tive consequences and expectancies in music [21], estab-
lishing clear connections. Evidence also suggests that some
auditory illusions, such as Binaural Auditory Beats, are as-
sociated with less negative mood, notably affecting perfor-
mance in vigilance tasks of sustained concentration, par-
ticularly when they occur in the beta range (16 and 24
hz) [22].
Regarding the concept of musical interaction being per-
ceived as rewarding, the Orff approach states that improvi-
sation with pentatonic scales, with bars and notes removed,
is always satisfactory and encourages freedom from the
fear of making mistakes [23]. The goal is for everybody
to experience success and appreciate the aesthetic in music
almost immediately, rather than having to learn notes and
rhythms before anything satisfying is accomplished. Per-
cussive patterns were important in his theory, as they ap-
peal to a primal essence in human beings and allow tempo
and beat to be experienced rather than learned. We in-
corporated many of these ideas into our work while de-
signing the response module and its implementation in a
music-making creative activity that can link cognitive fea-
tures and physical movements together to pay attention for
immediate responses. This is a generic hypothesis of re-
search in cognitive neuroscience of music and the concept
of music, mind and body couplings that our project takes
into account [2, 24]. The former points out the benefits of
music as a therapeutic tool and it can improve recovery in
the domains of verbal memory and focused attention. The
latter indicates its impact on social and individual develop-
ment through active engagement with music.
As mentioned, NOISA (Network Of Intelligent Sonic Agents)
is the robotic musical interface that we took into use and
built on for this research work. NOISA consists of three in-
struments for music creation, a Microsoft Kinect 2 camera
and a central computer. The three instruments are phys-
ically identical from each other. Each instrument is built
inside a plastic box. The interface for playing each in-
strument consists of two handles designed to provide an
ergonomic form for grasping them from several directions,
which can also position themselves actively, thanks to DC
motors (the robotic interactions with the physical medium).
The handles are attached to a motorised fader, which served
both as position indicator and as active position controller.
The hardware in each box includes an Arduino, a Rasp-
berry Pi, two motorised faders with extra large motors, two
capacitive touch sensors on the handles, and an embedded
speaker. In each instrument, sound production is largely
based on granular synthesis and phase vocoder algorithms
de-fragmenting radically small fragments of pre-existing
pieces from the classical music repertoire.
Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland
3.1 Response Module
Based on Huron’s ITPRA theory, we concluded that work-
ing in a stronger Prediction-Response musical connection
would likely produce an affective response evaluated as
positive [21]. Huron also proposes that recognisable pat-
terns are a common characteristic in music associated to
positive emotional responses. Thus, we decided that for
the current iteration of NOISA, the system would recog-
nise, store and playback musical patterns of 2 seconds of
activity, which the performer could easily identify based
on the data inputted. In order to avoid to solely repro-
duce an exact copy of the gesture performed by the per-
former, we also programmed a set of variations, based on
the music composition technique of motivic-through com-
position, also known as thematic development. This tech-
nique has been used successfully in classical music from
the common practice period (barroque to late XIX cen-
tury), and also in post-millennial music, including main-
stream genres [25].
We defined the gesture by recording a maximum of seven-
second long movements. The data recorded included the
engagement value, device identification, energy (measured
in decibels), spectral centroid (average of most prominent
frequency ranges over time), movement of the handlers and
time-stamps. Recording was initiated when a new move-
ment was detected, and stopped when inactive for longer
than one second. The gesture playback was activated de-
pending on the performer’s engagement in the task, activat-
ing more often when the engagement was low. The play-
back gestures were then chosen from a fifteen-minute inter-
val. Gesture data was measured against weights that were
defined by the energy mean value, spectral centroid, spec-
tral energy and amount of time elapsed since the recording
was made. The result was that, when the engagement was
higher, a larger set of chosen gestures was used. When
lower, newer patterns were chosen matching the readings
from the performers currently playing, to make the pattern
more recognisable.
The first choice of instrument for playback (out of the
three autonomous sonic agents) was the instrument on which
the gesture to be reproduced had been recorded. If that in-
strument was in use at that time, one of the other two in-
struments was randomly selected instead. There were four
variations of response playback: the recording itself, an
inverse (negative), a reverse and an inversed-reversed ver-
sion of the recording. The inverse was mapped into the
right range with the help of the highest and the lowest val-
ues. The engagement level of the performer was the de-
ciding factor for the chosen gesture variation. When less
engaged, the pure recording was the most common choice.
When engagement became higher, the performer experi-
enced a larger set of variations.
3.2 Hardware Design
The new response module required more precision than the
initial NOISA design could handle, so a new inner struc-
ture for the instrument was designed. Most of the hardware
parts were custom made with 3D printing, computer nu-
merical control (CNC) milling and laser cutting (see Fig-
Figure 1. New interface design of the NOISA instrument
ure 1). After a few iterations we reached the current state.
Aluminium tubes were added to make the handles stronger
yet light. The slide-bearings for the aluminium tubes were
CNC-milled for an exact fit. The handles were redesigned
to manipulate the added weight, using additional timing
belts and aluminium pulleys. The benefit of these changes
was lower friction from the handles. For the outer shell
to be strong enough to hold the new structure, we added
supporting layers and attached everything together with
custom-made corner-irons. The handles of the instrument
were also redesigned, with holes for the aluminium rods
and added space for the pressure sensors. To detect the
pressure, two pressure-sensitive conductive sheets were sand-
wiched between layers of copper tape. By reading the
change of the resistance, the amount of pressure could be
To review our main research goals in this work, we wanted
to observe, first, if participating in the skilled activity of
improvising music with our accompanying robotic instru-
ment showed any signs of being subjectively beneficial for
people. Such outcome would suggest that our improvi-
sation activity might be eligible as one of the many pos-
sible skilled tasks in the scope of this study (see again
our discussion about this in the introduction). Second, we
wanted to observe our interactive system in action, both
monitoring the levels of engagement with the activity (a
functionality already validated for earlier versions of this
instrument), and, more crucially, intervening (i.e., coach-
ing) when the performer’s level of engagement dropped to
low levels. As the main technical contribution, we wanted
to observe if the system was able to refocus the flow of the
performer’s activity until higher engagement levels were
again measured.
To do all of that, we designed and conducted a two-part
user study in which 15 voluntary participants (8 females)
took part. The first part aimed at recording a subjective
measure of wellbeing before and after a performance ses-
sion in which a person could improvise under normal con-
ditions. In the second part, in contrast, we provoked the
loss of focus of the participant in the improvisation activ-
Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland
Figure 2. Users developing the task
ity, and we observed if the system could observe the corre-
sponding drop in engagement, and if the instrument could
intervene collaboratively until engagement was recovered.
After a general description of the whole session and af-
ter obtaining consent, the first part of the user study started
with the participant filling out the PANAS questionnaire
(The Positive and Negative Affect Schedule) [26]. This is a
validated questionnaire that evaluates positive and negative
affects using a psychometric scale, and the change in those
within a specific period of time during which some treat-
ment activity took place (in our case, a performance ses-
sion with our collaborative musical instrument). PANAS
uses 10 descriptors per category of affects, showing rela-
tions between them with personality stats and traits. PANAS
has been successfully applied to measure overall psycho-
logical wellbeing before, and after a musically creative ac-
tivity [27]
The participant was then introduced to the NOISA instru-
ment, with explanations about how it works, followed by
a period of hands-on familiarisation with it (typically for
about three minutes). Once the participant felt confident
enough, the first music performance task was described,
which required performing a piece according to a some-
what restrictive set of rules (we wanted to reduce variabil-
ity in the performing activities at this stage in the study). In
particular, the participant would be asked to start perform-
ing with a specific instrument (Agent 1), then repeat the
given gesture with the middle instrument (Agent 2), finish-
ing with inputting the same repeated movement in the third
instrument (Agent 3), resulting in a piece that lasted for be-
tween three and five minutes. Once this was done, NOISA
would have typically started to make automatic responses.
At this moment, the participant was asked to explore freely
any of the agents, and once the participant was ready to fin-
ish, she/he had to manipulate the handles in Agent 1. The
participant was required to stop any input activity as an in-
dication of the end of the music task.
After finishing the task, the participant was asked to fill
in again the PANAS Questionnaire in order to compare
the impact of the task in the overall sense of psychologi-
cal wellbeing. Following that, the participant was asked to
evaluate the system’s usability by filling in the online ver-
sion of the System Usability Scale (SUS) designed by our
research group to facilitate the data gathering and analy-
sis. SUS has ”provided with that high-level measurement
of subjective usability, and over the years the work done by
other researchers has indicated that it is a valid and reliable
tool” [28].
For the second part of the study (designed to observe
the engagement tracking and recovery functionalities of
our interactive system), we asked the participant to repeat
the same musical task, extending it for a period of four
minutes. This time we tracked and logged the level of
engagement over time. We scheduled two events suffi-
ciently distracting at 1:30 (door opening, person coming
in and loudly manipulating a cardboard box) and a more
sublte event at 3:00 (phone ringing). We drew inspiration
from previous studies measuring the impact of distractions
in psychological wellbeing during a skilled activity [29].
The participant was not aware that such events would take
place, nor he/she was prepared for them. The total duration
of the test was approximately 30 minutes (see Figure 2).
The PANAS scores are divided in Positive Affects (PA)
and Negative Affects (NA). According to the PANAS doc-
umentation, the mean PA scores for momentary (immedi-
ate) evaluation are 29.7 (SD = 7.9). Scores can range from
10 to 50, with higher scores representing higher levels of
positive affect. For Negative Affects the mean NA score
for an equally momentary evaluation are 14.8 (SD = 5.4).
Scores can range from 10 to 50, with lower scores repre-
senting lower levels of negative affects.
During our tests, we obtained a PA score mean (Before
the activity) = 30.57 (SD = 3.89) and a PA score mean
(After the activity) = 34.21 (SD = 5.14). In addition, we
collected a NA Score (Before the activity) = 15.57 (SD =
6) and a NA Score (After the activity) = 12.30 (SD = 3.30).
This means an overall improvement of 3.64 PA points and
3.27 NA points. By adding PA and NA scores, we found
that the overall improvement in the reported psychological
wellbeing was of 6,91 points, which in a range of 10 to
50 in the PANAS range represents an overall improvement
of 17.27% after 3 minutes of activity with our system. In
addition, 30% of the participants reported an improvement
of more than 10 points in the PA affect score
In our paired before/after test, for Negative Affects the
two-tailed p value equals 0.0029 (t = 3.5979), a statistically
significant difference (significance level >99%). Regard-
ing the Positive Affects, we obtained a p value of 0.0699
(t = 1.9750), which is also statistically significant (signifi-
cance level >95%).
Regarding usability, the System Usability Scale research
Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland
drew the average of 68 points in the scale (from 0 to 100).
Sauro [30] found that when people rate a system or prod-
uct with a SUS score of 82, they are likely to recommend
a system or product to a friend or colleague (Net Promoter
Score). After evaluating our system, the participants gave
it a SUS mean value of 73.9 (SD = 11.33) which means
5.9 points above average. One third of the participants
gave our system a SUS score of 82 or higher, potentially
becoming a “promoter”.
Finally, our engagement graphs over time showed that
the system was able to identify and measure a distraction
event in 73.33% of the participants, impacting the overall
engagement by 0.2 points or more, in a scale of 0 (total
disengaged) to 1 (full engagement). Our engagement scale
is also divided into categories, where 0 to 0.3 means low
engagement, 0.3 to 0.6 medium engagement and 0.6 to 1
high engagement. Calculating every time a distraction had
an impact in the engagement, there was an overall average
of 0.41 sudden decrease (SD = 1.40). As the NOISA sys-
tem reacts once it detects a drop in the overall engagement,
the level of focus was effectively regained to a stable level
before the distraction event in a mean time of 10.07 sec-
onds (SD = 7.15). Two engagement over time graphs for
the participants 10 and 11 (see Figure 3 and 4) are included
to demonstrate an example of a case where our engage-
ment prediction system detected both distractions (90 and
180 second mark, respectively), managing to regain focus
of the participant through autonomous counteractions.
Figure 3. Engagement over time graph of the participant
Figure 4. Engagement over time graph of the participant
The results presented above show that the design of the
response module provided beneficial results for the partic-
ipants. The response module could successfully maintain
and strengthen the level of engagement in a skilled activity
of musical improvisation with NOISA instruments. The
resulting experience of such interaction was also likely to
have contributed positively to the psychological wellbeing
of the participants.
Judging from the quantitative data obtained through the
PANAS questionnaire, the average levels of Positive Af-
fects were higher after finishing the test, and the Nega-
tive Effects were lower after completing the task. In case
of Negative Affects, the difference was found to be very
statistically significant. In the PANAS scale, the Negative
Affects scores decreased from above the mean (15.57) to
below it (12.30). Even though the impact on the Positive
Affects was found to be not quite statistically significant,
we consider the overall improvement of 17.27% positive
considering the short amount of time spent with the system
during our task. The creative activity facilitated by NOISA
system proved to be statistically significant to impact on
the Negative Affects described in PANAS (Distressed, Up-
set, Guilty, Hostile, Irritable, Ashamed, Nervous, Jittery
and Afraid), which shows that the musical task showed
some capabilities during the test that can be successfully
applied to help reducing the subjective perception of worry,
stress and other negative personality traits of the partici-
The engagement estimation results indicated effective-
ness when detecting and maintaining the focus level of the
participants. In that sense, being able to measure at least
one of the distractions in 73.33% of the participants is sat-
isfactory, considering that while being observed, the par-
ticipant could feel compelled to keep his/her attention to
the task at hand. This situation would avoid us to detect
any distractions in their physical body movements. The
response module was capable of recovering attention and
focus from the participants in an average time of 10.07 sec-
onds after the distractions. We can note here that our cur-
rent contribution to NOISA system provided an engaging
interaction that was gradually and progressively built up by
the participant’s active and continuous involvement in the
musical activity with NOISA instruments.
Similarly, we observed that both our new hardware de-
sign and response module were evaluated by the partici-
pants as easy to use, obtaining consistent above-average
scores in the SUS scale. In addition, a large portion (one
third) of the participants cataloged it above the 82 mark,
which is particularly positive considering that we asked
non-specialist individuals to evaluate a new musical inter-
face. Our sample consisting of 15 participants might be
considered small; however, Tullis and Stetson [31] have
already demonstrated the reliability of SUS in smaller sam-
ples (8-12 participants).
Based on previous literature, it is clear that designing for
wellbeing is strongly linked with the psychological under-
standing of wellbeing, positive experience and physical ac-
tivity [3]. The purpose of our project was to establish a
situated musical activity for the user to be engaged in, at-
tempting to achieve the goal of enhancing psychological
Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland
wellbeing. Our design was grounde in a concrete con-
text (creative musical overall process) and guided by the
functionality (hand-held interface) as well as through sub-
tle interaction (counter-responsive musical actions). We
observed and measured the behaviour of our system as
it deals with losses of performer focus provoked by the
controlled introduction of external distractors. During the
tests, we observed indications that being engaged in our
musical activity contributed positively to participants psy-
chological wellbeing. Encouraged by our initial results, we
plan to continue our research by including in a future study
a comparison with a control group, which will enable us to
draw in a longer term more comprehensive conclusions of
the impact of the activity with NOISA in the pyschological
wellbeing of the user.
We also aim to investigate the application of a similar
engagement-monitoring technique in a broader scope of
skilled, potentially mentally absorbing actions with peo-
ple working in different occupations. A foreseen psycho-
logical wellbeing application scenario that will be imple-
mented in the future phases of the project, could be the
non-intrusive recognition of changes in patterns of engage-
ment that a person presents when carrying out different
creative tasks with digital environments, which could sig-
nal changes in their mood, psychological health or deteri-
oration of their cognitive capacity (early diagnostic goal).
This application requires a new hand-held interface to be
designed and implemented which will be fully integrated
with the engaging monitoring and the new response mod-
ule in NOISA system. Through subtle external actions, the
new interface and the digital environment could assist the
person to regain focus and engagement with the creative
task in hand, which might be beneficial for the success-
ful completion of the task (assistive living goal), and to
do so with an enhanced sense of control, accomplishment
and satisfaction (psychological wellbeing goal) in a cre-
ative process.
This project was developed thanks to the support of the
Nokia Research Centre in Finland.
[1] D. H. Pink, Drive: The surprising truth about what
motivates us. Penguin, 2011.
[2] C. D. Ryff, “Psychological well-being in adult life,”
Current directions in psychological science, vol. 4,
no. 4, pp. 99–104, 1995.
[3] S. Diefenbach, M. Hassenzahl, K. Eckoldt, L. Hartung,
E. Lenz, and M. Laschke, “Designing for well-being:
A case study of keeping small secrets,The Journal of
Positive Psychology, pp. 1–8, 2016.
[4] K. Tahiro˘
glu, T. Svedstr¨
om, and V. Wikstr¨
“Noisa: A novel intelligent system facilitating smart
interaction,” in Proceedings of the 33rd Annual
ACM Conference Extended Abstracts on Human
Factors in Computing Systems, ser. CHI EA ’15.
ACM, 2015, pp. 279–282. [Online]. Available:
[5] K. Tahiroglu, T. Svedstr ¨
om, and V. Wikstr¨
“Musical engagement that is predicated on intentional
activity of the performer with noisa instruments,
in Proceedings of the International Conference on
New Interfaces for Musical Expression, E. Berdahl
and J. Allison, Eds., May 31 – June 3 2015, pp.
132–135. [Online]. Available:
proceedings/2015/nime2015 121.pdf
[6] J. J. Appleton, S. L. Christenson, D. Kim, and A. L.
Reschly, “Measuring cognitive and psychological
engagement: Validation of the student engagement
instrument,” Journal of School Psychology, vol. 44,
no. 5, pp. 427 – 445, 2006, motivation. [Online]. Avail-
[7] M. L. Maehr and H. A. Meyer, “Understanding
motivation and schooling: Where we’ve been, where
we are, and where we need to go, Educational
Psychology Review, vol. 9, no. 4, pp. 371–409,
1997. [Online]. Available:
[8] J. M. Reeve, H. Jang, D. Carrell, S. Jeon, and
J. Barch, “Enhancing students’ engagement by
increasing teachers’ autonomy support,” Motivation
and Emotion, vol. 28, no. 2, pp. 147–169, 2004.
[Online]. Available:
[9] E. Frydenberg, M. Ainley, and V. J. Rus-
sell, “Student motivation and engagement,
Schooling issues digest ; 2005/2, 2005, 16p.
[Online]. Available:
sectors/school education/publications resources/
schooling issues digest/documents/schooling issues
digest motivation engagement pdf.htm
[10] N. Bryan-Kinns and P. G. Healey, “Decay in
collaborative music making, in Proceedings of
the International Conference on New Interfaces
for Musical Expression, Paris, France, 2006, pp.
114–117. [Online]. Available:
proceedings/2006/nime2006 114.pdf
[11] G. Leslie, “Measuring musical engagement,” UC San
Diego: Music, 2013.
[12] M. E. Barrera, M. H. Rykov, and S. L. Doyle, “The ef-
fects of interactive music therapy on hospitalized chil-
dren with cancer: a pilot study,Psycho-Oncology,
vol. 11, no. 5, pp. 379–388, 2002.
[13] J. M. Standley and S. B. Hanser, “Music therapy re-
search and applications in pediatric oncology treat-
ment,” Journal of Pediatric Oncology Nursing, vol. 12,
no. 1, pp. 3–8, 1995.
Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland
[14] P. Keyani, G. Hsieh, B. Mutlu, M. Easterday, and
J. Forlizzi, “Dancealong: supporting positive social ex-
change and exercise for the elderly through dance,” in
CHI’05 extended abstracts on Human factors in com-
puting systems. ACM, 2005, pp. 1541–1544.
[15] J. Music and R. Murray-Smith, “Virtual hooping:
teaching a phone about hula-hooping for fitness, fun
and rehabilitation,” in Proceedings of the 12th interna-
tional conference on Human computer interaction with
mobile devices and services. ACM, 2010, pp. 309–
[16] P. Riley, N. Alm, and A. Newell, “An interactive tool
to promote musical creativity in people with dementia,
Computers in Human Behavior, vol. 25, no. 3, pp. 599–
608, 2009.
[17] K. Stensæth and E. Ruud, “An interactive technology
for health: New possibilities for the field of music and
health and for music therapy? a case study of two chil-
dren with disabilities playing with ”orfi”,” Series from
the Centre for Music and Health, vol. 8, no. 7, pp. 39–
66, 2014.
[18] A. Gabrielsson and P. N. Juslin, Emotional expression
in music. Oxford University Press, 2003.
[19] P. N. Juslin and P. Laukka, “Expression, perception,
and induction of musical emotions: A review and a
questionnaire study of everyday listening,Journal of
New Music Research, vol. 33, no. 3, pp. 217–238,
[20] L.-L. Balkwill, W. F. Thompson, and R. Matsunaga,
“Recognition of emotion in japanese, western, and hin-
dustani music by japanese listeners,” Japanese Psycho-
logical Research, vol. 46, no. 4, pp. 337–349, 2004.
[21] D. Huron, Sweet anticipation: Music and the psychol-
ogy of expectation. MIT press, 2006.
[22] J. D. Lane, S. J. Kasian, J. E. Owens, and G. R. Marsh,
“Binaural auditory beats affect vigilance performance
and mood,” Physiology & behavior, vol. 63, no. 2, pp.
249–252, 1998.
[23] A. Long, “Involve me: Using the orff approach within
the elementary classroom,” Awards for Excellence in
Student Research and Creative Activity, no. 4, 2013.
[24] S. Hallam, “The power of music: Its impact on the in-
tellectual, social and personal development of children
and young people,” International Journal of Music Ed-
ucation, vol. 28, no. 3, pp. 269–289, 2010.
[25] B. Osborn, “Understanding through-composition in
post-rock, math-metal, and other post-millennial rock
genres,” Music Theory Online, vol. 17, no. 3, 2011.
[26] D. Watson, L. A. Clark, and A. Tellegen, “Develop-
ment and validation of brief measures of positive and
negative affect: the panas scales.” Journal of personal-
ity and social psychology, vol. 54, no. 6, p. 1063, 1988.
[27] A. M. Sanal and S. Gorsev, “Psychological and phys-
iological effects of singing in a choir,Psychology of
Music, vol. 42, no. 3, pp. 420–429, 2014.
[28] J. Brooke, “Sus: a retrospective,” Journal of usability
studies, vol. 8, no. 2, pp. 29–40, 2013.
[29] B. Thorgaard, B. B. Henriksen, G. Pedersbæk, and
I. Thomsen, “Specially selected music in the cardiac
laboratory - an important tool for improvement of the
wellbeing of patients,” European Journal of Cardio-
vascular Nursing, vol. 3, no. 1, pp. 21–26, 2004.
[30] J. Sauro, “Measuring usability with the system usabil-
ity scale (sus). 2011,” Available at: http://www. mea-
suringusability. com/sus. php [22.04. 2013], 2015.
[31] T. S. Tullis and J. N. Stetson, “A comparison of ques-
tionnaires for assessing website usability,” in Usability
Professional Association Conference, 2004, pp. 1–12.
Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
To a good part, well-being depends on individual engagement in beneficial activities. The present paper draws attention to the potential of encouraging and shaping well-being-enhancing activities through interaction with everyday objects and technology. Our case study started from the activity of keeping ‘small’ secrets as a way to experience autonomy and privacy. We situated this activity in an office and ‘materialized’ it through a digital picture frame, holding a secret picture. Inspired by seven interviewees’ detailed descriptions of interacting with secrets, we designed an especially secretive interaction to consume the picture and compared it to a more technical interaction. In a first empirical exploration, using video prototypes (N = 276), the secretive interaction was rated as more positive and providing more intense feelings of privacy and autonomy. This hints at the potential of objects and the careful design of interaction with this objects to intensify the positive experiences gained from mundane activities.
Full-text available
Conference Paper
Full-text available
This paper presents our current research in which we study the notion of performer engagement within the variance and diversities of the intentional activities of the performer in musical interaction. We introduce a user-test study with the aim to evaluate our system's engagement prediction capability and to understand in detail the system's response behaviour. The quantitative results indicate that our system recognises and monitors performer's engagement successfully , although we found that the system's response to maintain and deepen the performer's engagement is perceived differently among participants. The results reported in this paper can be used to inform the design of interactive systems that enhance the quality of performer's engagement in musical interaction with new interfaces.
Conference Paper
Full-text available
In this paper, we describe NOISA (Network of Intelligent Sonic Agents). NOISA is an intelligent system that acts to maintain and deepen the user's engagement with digital artefacts by learning from the user's actions and behavioural patterns in the moment of interaction. It facilitates a smart interaction by monitoring user's bodily movements, facial expressions and control inputs. We present our model and system in a musical context, interfaced with our digital musical instrument (DMI). Our concept can be further extended to possible application areas in Human Computer Interaction (HCI) research field.
This randomized controlled trial aimed to quantify the effects of choir singing on emotional state and anxiety levels of singers. Salivary amylase, PANAS (Positive and Negative Affect Schedule) and STAI-s/-t(State-Trait Anxiety Inventory) were applied before and after a 1-hour single choir session for an experimental group (n = 35) and unstructured time for control group (n = 35). Amylase decreased in the experimental group but increased in the control group (p > 0.05 for Fgroup; p = 0.014 for Ftestbygroup). Follow-up analysis showed this interaction to be due to baseline differences between the two groups. Negative affect decreased in the experimental group and increased in the control group (p > 0.05 for Ftest and Fgroup; p = 0.006 for Ftest by group). A decrease in positive affect was found between the pre- and post-test for the control group (p = 0.023 for Ftest; p = 0.004 for Ftest by group). State anxiety decreased in the experimental group and increased in the control group (p > 0.05 for Ftest and Fgroup; p = 0.001 for Ftest by group). Singing in a choir was found to have a positive impact on psychological indicators of affect and anxiety, however, its physiological effect could not be shown using salivary amylase in this study.
Since the dawn of experimental rock’s second coming in the new millennium, experimental artists have begun distancing themselves from Top-40 artists through formal structures that eschew recapitulatory verse/chorus conventions altogether. In order to understand the correlation between genre and form more thoroughly, this paper provides a taxonomic approach to through-composition in several post-millennial experimental rock genres including post-rock, math-metal, art rock, and neo-prog. Combining the presence or absence of two salient formal parameters (hierarchical grouping structure and thematic unification) generates four possible through-composed archetypes. Representative examples from the post-millennial rock corpus are provided for each archetype, and accompanying analyses identify the specific musical elements that engender such formal divisions. Included interviews suggest that this taxonomic model is congruent with the language musicians use to describe their own compositional strategies. Throughout the article, a case is made for the substantive link between specific types of through-composed forms and the genres in which those forms regularly appear, as well as for the difference between formal designs found in post-millennial experimental rock and those found in conventional rock music.
This paper presents an interpretive summary of the progress that has been made in the study of motivation, especially as this work relates to the area of education. Motivation research has “come of age” with theoretical frameworks rooted in an established body of findings, and with the development of core methodologies for pursuing further knowledge. It has begun to provide a firm base for guiding educational practice. Current work is clearly in accord with mainstream psychology in that cognitive models of motivation are the predominant guides for research. Future work is likely to focus increasingly on how the meaning construction process affects engagement in tasks. Researchers should consider a wider array of research procedures and give greater attention to understanding motivation as it occurs in natural settings, especially as educational interventions are attempted. A serious deficiency in the motivation literature is that relatively little attention has been given to differences related to sociocultural backgrounds.
Knowledge of psychological well-being persistently lags behind knowledge of psychological dysfunction. The imbalance is evident in magnitude of research-studies of psychological problems dwarf the literature on positive psychological functioning-and in the meaning of basic terms (e.g., typical usage equates health with the absence of illness). A person is viewed as mentally sound if he or she does not suffer from anxiety, depression, or other forms of psychological symptomatology. This prevailing formulation never gets to the heart of wellness; to do so, we must define mental health as the presence of the positive. To explicate the positive is, however, to grapple with basic values and ideals of the human experience. These values are no less evident in definitions of human suffering, although consensus in identification of the negative is somehow easier to achieve. Despite these challenges, much has been written, within the field of psychology and outside it, regarding the contours of positive psychological functioning.