Conference PaperPDF Available

Intelligent Music Interfaces: When Interactive Assistance and Adaptive Augmentation Meet Musical Instruments

Authors:
Intelligent Music Interfaces: When Interactive Assistance and
Adaptive Augmentation Meet Musical Instruments
Thomas Kosch
HU Berlin
Berlin, Germany
thomas.kosch@hu-berlin.de
Andreas Weiß
Musikschule Schallkultur
Kaiserslautern, Germany
andreas.weiss@musikschule-
schallkultur.de
Jordan Aiko Deja
University of Primorska
Koper, Slovenia
De La Salle University
Manila, Philippines
jordan.deja@dlsu.edu.ph
Snehesh Shrestha
University of Maryland
College Park, USA
snehesh@umd.edu
Matthias Hoppe
LMU Munich
Munich, Germany
matthias.hoppe@i.lmu.de
Andrii Matviienko
KTH Royal Institute of Technology
Stockholm, Sweden
andriim@kth.se
Karola Marky
Ruhr University Bochum
Bochum, Germany
karola.marky@rub.de
ABSTRACT
The interactive augmentation of musical instruments to foster self-
expression and learning has a rich history. Over the past decades,
incorporating interactive technologies into musical instruments has
emerged as a research eld requiring strong collaboration between
disciplines. The workshop “Intelligent Music Interfaces” covers a
wide range of musical research subjects and directions, including (a)
current challenges in musical learning, (b) prototyping for improve-
ments, (c) new means of musical expression, and (d) evaluation of
the solutions.
CCS CONCEPTS
Human-centered computing
Interactive systems and
tools;Interaction techniques;Interaction devices.
KEYWORDS
Music Interfaces, Musical Instruments, Self-Expression, Augmented
Instruments
ACM Reference Format:
Thomas Kosch, Andreas Weiß, Jordan Aiko Deja, Snehesh Shrestha, Matthias
Hoppe, Andrii Matviienko, and Karola Marky. 2024. Intelligent Music In-
terfaces: When Interactive Assistance and Adaptive Augmentation Meet
Musical Instruments. In The Augmented Humans International Conference
(AHs 2024), April 04–06, 2024, Melbourne, VIC, Australia. ACM, New York,
NY, USA, 4 pages. https://doi.org/10.1145/3652920.3653039
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
AHs 2024, April 04–06, 2024, Melbourne, VIC, Australia
©2024 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-0980-7/24/04
https://doi.org/10.1145/3652920.3653039
1 BACKGROUND & RELEVANCE TO THE
CONFERENCE
Since the rst musical expressions of humans over 42,000 years ago,
countless ways of creating and experiencing sound have been cre-
ated and rened. Today, new sensors and computational capabilities
of musical instruments are leveraged to broaden the artists’ expres-
siveness [
11
,
20
], enhance teaching scenarios [
9
,
10
,
18
] or allow
remote collaboration of musicians. Ultimately, intelligent musical
interfaces can augment humans’ capabilities to express themselves
by creating sound and music.
Initial work in the domain of intelligent music interfaces focused
on improving the play performance of students through learning-
by-demonstration [
12
,
17
,
22
] or by reecting the performance di-
rectly to the student for real-time improvements [
3
,
8
,
15
]. Further,
musical instruments were augmented by technologies to extend
the musical sound space. For example, gestures and musical instru-
ments can be combined to change the pitch of a sound [
7
,
11
,
20
].
We expect future musical instruments to integrate interactive fea-
tures, promote self-expression, change stage performances, and
augment the audience’s perception of a performance [
6
]. Conse-
quently, intelligent musical interfaces are highly connected to hu-
man augmentation. Not only is the creation of music and sound
augmented, but so is the experience of it. The second edition of the
IMI workshop was held in 2023 in conjunction with the AHs ’23
conference [2].
2 WORKSHOP CONTENT
The workshop “Intelligent Music Interfaces” (IMI) lays the founda-
tion for a research eld concerning integrating interactive compo-
nents into musical instruments for creating new ways of musical
expression and creating immersive performances for audiences.
We aim to connect recent research revolving in this eld with the
workshop to start, grow, and foster a community around intelligent
musical interfaces. This includes presenting, demonstrating, and
327
AHs 2024, April 04–06, 2024, Melbourne, VIC, Australia Kosch et al.
discussing existing, augmented, and novel musical instruments and
technology used during stage performances.
Our workshop oers various research topics in (a) new ways of
musical expression and perception, (b) prototyping for improve-
ments, and (c) evaluation of the solutions. Musical expertise is
optional for participation in the workshop since we aim to connect
researchers from dierent disciplines.
We plan an in-person workshop led by 1-2 of the organizers, but
we are prepared to switch to full virtual participation if the circum-
stances do not allow physical participation. Either way, we intend
to oer an inclusive format and enable remote participation via
Zoom in case special circumstances prevent in-person participation.
Details for that are given in the workshop structure below; details
for the submission formats are provided in the call for participa-
tion. We plan interactive sessions where participants can engage
with several intelligent music interfaces to foster discussion. The
exhibits are from workshop participants and the organizers. The
organizers will ensure that there are enough exhibits to ll this
workshop. We need one projector and several power sockets for
the participants and demos.
3 GOALS AND OUTCOME
IMI brings together researchers, artists, and practitioners from
dierent disciplines to foster interdisciplinary research. After the
workshop, we encourage researchers to rework their publications
based on the discussions and feedback from the workshop. We will
support researchers in submitting their nal papers to arXiv or
as preprints on our website. Based on the group work and moder-
ated discussion, the organizers plan to distill critical aspects and
the workshop’s outcomes into a position paper published open
access. The anticipated results are available to research questions
concerning prototyping, the study design, and the evaluation of
intelligent music interfaces. The feedback from the workshop atten-
dees accompanies these research questions to inspire researchers
interested in tackling the research questions. Based on the interest
of the workshop attendees, we organize regular meetups. We plan
to establish a long-term format with a potential future invitation
for the authors to contribute to a journal.
4 SCHEDULE
Pre-Workshop Plans: We will distribute information and materi-
als on our workshop website. Information includes the intention,
motivation, and potential outcomes of the workshop. Furthermore,
the website serves as a platform to advertise and acquire potential
workshop participants. The workshop website will be available
under the domain snehesh.com/imi/. The website includes a work-
shop description, objectives, and possible submission topics. It also
hosts the call for participation, a link to the submission system,
the workshop schedule, further organizational information, and
information about the workshop organizers. Accepted papers will
be made publicly available on the website before the conference to
maximize the preparation time for the workshop and foster discus-
sions. Finally, workshop participants can join our Slack channel to
receive updates about the workshop and join our community.
Workshop Plan: We plan a full-day workshop for around 20 par-
ticipants and the following schedule:
(1)
Workshop introduction (15 min): the organizers introduce
themselves, the workshop topic, and the schedule.
(2)
Moderated speed dating (approx. 15 min): the workshop
attendees participate in speed dating sessions to get to know
each other by physically grouping them.
(3)
Introduction of interactive session (10 min): the organiz-
ers introduce the interactive session and answer questions.
Interactive presentations and demonstrations will be set up
before the workshop.
(4)
Interactive music session (60 min): hands-on experience
with dierent intelligent music interfaces for the partici-
pants.
(5) Short break
(6)
Keynote (20 min + 10 min discussion): the keynote speaker
will be announced one week before the workshop. For the
keynote, we requested the German tech journalist Pina Merk-
ert1who does creative tech projects.
(7) Lunch break
(8)
Art Pieces (approx. 3
×
10 min): participants perform their
art piece.
(9) Short break
(10)
Pitch presentations of short papers and research state-
ments (total 70 min): 5 Research Statements 3+2 min (25
min), short break (5 min), and 6 Short Paper 5+2 minutes (42
min).
(11) Coee break
(12)
Moderated discussion and closing (60 min): the organiz-
ers moderate a discussion based on the pitch presentations,
art pieces, and interactive demonstrations. Finally, the work-
shop is closed.
5 RECRUITMENT & REVIEWING
The organizers use their social networks and mailing lists to dis-
seminate the call for participation (see below). Submissions will be
collected via EasyChair and reviewed by the IMI PC. Each submis-
sion will receive 2-3 reviews from the PC members and external
reviewers.
Call for Participation: Playing a musical instrument has many
benets, such as positively impacting mental health or dexterity.
Electronic elements were integrated into traditional musical instru-
ments in the early 1930s to create instruments, such as E-guitars,
that oer new musical expression. Electric instruments evolved by
combining networking and computational capabilities. These new
capabilities can further broaden artists’ expressiveness, enhance
learning scenarios, allow musicians to collaborate remotely, and
create new musical instruments.
In this workshop, we will discuss and interact with intelligent
music interfaces of any form. Novel music interfaces could be a
new adaption of a traditional musical instrument, an interface for
learning, or even supporting software. The workshop will be held
in person with the Augmented Humans International Conference
1https://www.heise.de/autor/pina-merkert-4265420 accessed 20-March-2024
328
Intelligent Music Interfaces AHs 2024, April 04–06, 2024, Melbourne, VIC, Australia
on April 4th in Melbourne, Australia, while oering to participate
virtually.
Submissions should follow the ACM two-column format with
a length between two and four pages, excluding references. We
solicit the following types of submissions: position papers,research
statements,art pieces, and interactive demonstrations. The duration
of an art piece is limited to 15 minutes. As interactive demonstra-
tions, we consider demonstrating an intelligent musical interface
that workshop participants can try out during the workshop. The
authors of interactive demonstrations and art pieces are invited to
present a prototype in the interactive workshop session.
Information about submitting papers can be found on the work-
shop website
2
. Participants will be selected based on the merit of
their contribution to the workshop. We encourage authors to make
their research available on arXiv
3
after the workshop. At least one
author of each accepted submission must attend the workshop. All
participants must register for the workshop.
6 ORGANISER BIOGRAPHIES
The rst version of the IMI 2022 workshop was conducted in con-
junction with CHI 2022 in New Orleans [
16
]. The second version,
IMI 2023, took place during the AHs 2023 conference with great
success [
2
]. The organizing committee of the third IMI workshop
consists of the following researchers and professional musicians.
Each of them contributes long-term experiences in organizing work-
shops including Handling IoT in HCI (IoT ’17), Reading the Mobile
Brain (MUM ’17) [
5
], Designing Assistive Environments for Manu-
facturing (PETRA ’17 - ’21)
4
, SmartObjects ’18 (CHI ’18) [
21
] and
SmartObjects ’22 (ISS ’22) [
25
], a series of workshops and events
about vulnerable road users [
13
,
14
,
19
,
23
,
24
,
30
] as well as several
local workshops for bands and musicians.
Thomas Kosch is a professor at the Humboldt University of Berlin.
His research focuses on physiological interaction, including design-
ing, prototyping, and evaluating physiological user interfaces. In
addition, he is an expert in integrating physiological sensing into
musical instruments using EMG [
7
,
8
,
11
] or customized 3D-printed
sensors [
26
] to implicitly and explicitly augment musicians. Thomas
is deeply interested in new ways to create music, augment existing
instruments, and create tools and feedback mechanisms supporting
musical students. He will provide his experience in prototyping
and evaluating novel music interfaces. He has played piano, guitar,
drums, and alto saxophone for several years.
Andreas Weiß has over 30 years of experience as a musician, com-
poser, and music teacher. He is part of several band projects and
co-owns the music school Schallkultur in Kaiserslautern, Germany.
In addition, he collaborates with several research institutions by
contributing his expertise as a musician to develop and evaluate
new innovative music interfaces, such as Let’s Frets [
17
,
18
]. An-
dreas’ practical didactic experience will allow workshop attendees
to quickly identify challenges and opportunities when using inter-
active technologies in learning scenarios.
Snehesh Shrestha (https://snehesh.com) is a PhD candidate at the
University of Maryland (UMD) College Park. His research focuses
2https://www.thomaskosch.com/imi/ accessed 20-March-2024
3https://arxiv.org accessed 20-March-2024
4www.petrae.org/workshops/DAEM.html accessed 20-March-2024
on AI-assisted music education. He develops technology, tools, and
applications to provide real-time feedback during practice for music
players. He has an open-sourced fast video annotation tool, FEVA
[
28
], developed perception models such as high temporal resolu-
tion 3D human pose estimation, AIMusicGuru [
27
], and is currently
working on music understanding and feedback models. He is in-
terested in empowering music teachers by creating super-tools
that augment their capabilities in understanding their students’
strengths and weaknesses. He is also interested in developing tech-
nologies that foster good form and habits to avoid injuries and learn
better techniques. He has been playing the guitar for 25 years and
is currently learning to play the violin.
Matthias Hoppe is a PhD student at LMU Munich, where he
focuses on mixed reality as a new medium and investigates the
application of haptic feedback in virtual reality to alter one’s percep-
tion. Therefore, he is also interested in how such novel interactions
can enhance experiences with novel music interfaces. Matthias has
experience in evaluating supportive tools while practicing musical
instruments.
Jordan Aiko Deja (https://jrdndj.com) is a PhD candidate working
on augmented reality, music learning, and adaptive visualizations in
the HICUP research group at the University of Primorska, Slovenia.
He is researching augmented reality techniques to teach impro-
visation on the piano. His main goal is to design interfaces that
enable people to be more creative. He has also worked on several
instruments and interfaces such as the EMG-based guitar [29], Vi-
Tune for the Deaf and Hard-of-Hearing [
1
], and the ImproVISe for
the Piano [
4
]. He is also part of the Center for Complexity and
Emergencies Technologies (COMET) research group from De La
Salle University, Philippines.
Andrii Matviienko is an assistant professor at KTH Royal Insti-
tute of Technology in Stockholm, Sweden. His research focuses
on assisting technology in urban environments, mainly designing,
constructing, and evaluating multimodal and mixed-reality inter-
faces for vulnerable road users. Additionally, he has over 20 years of
experience playing trombone in amateur and semi-professional or-
chestras in Ukraine and Germany and a bass guitar in jazz/funk/rock
bands.
Karola Marky is a professor at the Ruhr-University Bochum. Her
research focuses on the self-determination and self-expression of
individuals in digital spaces, explicitly considering ubiquitous tech-
nology and novel (security and privacy) interfaces based on tangible
interaction. She leverages novel interfaces and interaction tech-
niques to improve musical instruments dedicated to beginners and
students (cf. [
18
]). In her free time, she plays the piano and sings.
She will coordinate the workshop organization and contribute with
her expertise in evaluating novel intelligent music interfaces.
REFERENCES
[1]
Jordan Aiko Deja, Alexczar Dela Torre,Hans Joshua Lee, Jose Florencio Ciriaco I V,
and Carlo Miguel Eroles. 2020. Vitune: A visualizer tool to allow the deaf and
hard of hearing to see music with their eyes. In Extended Abstracts of the 2020
CHI Conference on Human Factors in Computing Systems. 1–8.
[2]
Jordan Aiko Deja, Bettina Eska, Snehesh Shrestha, Matthias Hoppe, Jakob Karo-
lus, Thomas Kosch, Andrii Matviienko, Andreas Weiß, and Karola Marky. 2023.
Intelligent Music Interfaces: When Interactive Assistance and Augmentation
Meet Musical Instruments. In Proceedings of the Augmented Humans International
Conference 2023. 379–383.
329
AHs 2024, April 04–06, 2024, Melbourne, VIC, Australia Kosch et al.
[3]
Jordan Aiko Deja, Sven Mayer, Klen Čopič Pucihar, and Matjaž Kljun. 2022. A
Survey of Augmented Piano Prototypes: Has Augmentation Improved Learning
Experiences? Proceedings of the ACM on Human-Computer Interaction 6, ISS
(2022), 226–253.
[4]
Jordan Aiko Deja, Sandi Štor, Ilonka Pucihar, Klen Čopič Pucihar, and Matjaž
Kljun. 2024. Teach Me How to ImproVISe: Co-Designing an Augmented Piano
Training System for Improvisation, In 8th Human-Computer Interaction Slovenia
conference 2023. arXiv preprint arXiv:2402.02999.
[5]
Christiane Glatz, Jonas Ditz, Thomas Kosch, Albrecht Schmidt, Marie Lahmer,and
Lewis L. Chuang. 2017. Reading the Mobile Brain: From Laboratory to Real-World
Electroencephalography. In Proceedings of the 16th International Conference on
Mobile and Ubiquitous Multimedia (Stuttgart, Germany) (MUM ’17). Association
for Computing Machinery, New York, NY, USA, 573–579. https://doi.org/10.
1145/3152832.3156560
[6]
Jiawen Han, George Chernyshov, Moe Sugawa, Dingding Zheng, Danny Hynds,
Taichi Furukawa, Marcelo Padovani, Kouta Minamizawa, Karola Marky, Jamie A
Ward, and Kai Kunze. 2022. Linking Audience Physiology to Choreography.
ACM Trans. Comput.-Hum. Interact. 29 (aug 2022), 1–32. https://doi.org/10.1145/
3557887 Just Accepted.
[7]
Jakob Karolus, Annika Kilian, Thomas Kosch, Albrecht Schmidt, and Paweł W.
Wozniak. 2020. Hit the Thumb Jack! Using Electromyography to Augment the
Piano Keyboard. In Proceedings of the 2020 ACM Designing Interactive Systems
Conference (Eindhoven, Netherlands) (DIS ’20). Association for Computing Ma-
chinery, New York, NY, USA, 429–440. https://doi.org/10.1145/3357236.3395500
[8]
Jakob Karolus, Hendrik Schu, Thomas Kosch, Paweł W. Wozniak, and Albrecht
Schmidt. 2018. EMGuitar: Assisting Guitar Playing with Electromyography. In
Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong,
China) (DIS ’18). Association for Computing Machinery, New York, NY, USA,
651–655. https://doi.org/10.1145/3196709.3196803
[9]
Joseph R. Keebler, Travis J. Wiltshire, Dustin C. Smith, and Stephen M. Fiore. 2013.
Picking Up STEAM: Educational Implications for Teaching With an Augmented
Reality Guitar Learning System. In Proceedings of the International Conference
on Virtual, Augmented and Mixed Reality (VAMR). Springer, Cham, Switzerland,
170–178. https://doi.org/10.1007/978-3- 642-39420- 1_19
[10]
Chutisant Kerdvibulvech and Hideo Saito. 2007. Real-Time Guitar Chord Esti-
mation by Stereo Cameras for Supporting Guitarists. In Proceedings of the 10th
International Workshop on Advanced Image Technology (IWAIT). The Institute
of Electronics, Information and Communication Engineers, Bangkok, Thailand,
256–261.
[11]
Annika Kilian, Jakob Karolus, Thomas Kosch, Albrecht Schmidt, and Paweł W.
Paweł. 2021. EMPiano: Electromyographic Pitch Control on the Piano Keyboard.
In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing
Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery,
New York,N Y,USA, Article 196, 4 pages. https://doi.org/10.1145/3411763.3451556
[12]
Markus Löchtefeld, Sven Gehring, Ralf Jung, and Antonio Krüger. 2011. GuitAR:
Supporting Guitar Learning through Mobile Projection. In CHI ’11 Extended
Abstracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI
EA ’11). Association for Computing Machinery, New York, NY, USA, 1447–1452.
https://doi.org/10.1145/1979742.1979789
[13]
Andreas Löcken, Mark Colley, Andrii Matviienko, Kai Holländer, Debargha Dey,
Azra Habibovic, Andrew L Kun, Susanne Boll, and Andreas Riener. 2021. We-
CARe: Workshop on Inclusive Communication between Automated Vehicles and
Vulnerable Road Users. In 22nd International Conference on Human-Computer In-
teraction with Mobile Devices and Services (Oldenburg, Germany) (MobileHCI ’20).
Association for Computing Machinery, New York, NY, USA, Article 43, 5 pages.
https://doi.org/10.1145/3406324.3424587
[14]
Andreas Löcken, Andrii Matviienko, Mark Colley, Debargha Dey, Azra Habibovic,
Yee Mun Lee, and Andreas Riener. 2022. Accessible Automated Automotive
Workshop Series (A3WS):International Perspective on Inclusive External Human-
Machine Interfaces. In Adjunct Proceedings of the 14th International Conference on
Automotive User Interfaces and Interactive Vehicular Applications (Seoul, Republic
of Korea) (AutomotiveUI ’22). Association for Computing Machinery, New York,
NY, USA, 192–195. https://doi.org/10.1145/3544999.3551347
[15]
Karola Marky, Julian Fischer, Max Mühlhäuser, and Andrii Matviienko. 2021.
Investigating Page Turning Methods for Sheet Music during Piano Play. In
Adjunct Publication of the 23rd International Conference on Mobile Human-
Computer Interaction (Toulouse & Virtual, France) (MobileHCI ’21). Association
for Computing Machinery, New York, NY, USA, Article 18, 6 pages. https:
//doi.org/10.1145/3447527.3474863
[16]
Karola Marky, Annika Kilian, Andreas Weiß, Jakob Karolus, Matthias Hoppe,
Pawel W. Wozniak, Max Mühlhäuser, and Thomas Kosch. 2022. Intelligent
Music Interfaces: When Interactive Assistance and Augmentation Meet Musical
Instruments. In Extended Abstracts of the 2022 CHI Conference on Human Factors
in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for
Computing Machinery, New York, NY, USA, Article 84, 4 pages. https://doi.org/
10.1145/3491101.3503743
[17]
Karola Marky, Andreas Weiß, Andrii Matviienko, Florian Brandherm, Sebastian
Wolf, Martin Schmitz, Florian Krell, Florian Müller, Max Mühlhäuser, and Thomas
Kosch. 2021. Let’s Frets! Assisting Guitar Students During Practice via Capacitive
Sensing. In Proceedings of the 2021 CHI Conference on Human Factors in Computing
Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New
York, NY, USA, Article 746, 12 pages. https://doi.org/10.1145/3411764.3445595
[18]
Karola Marky, Andreas Weiß, Florian Müller, Martin Schmitz, Max Mühlhäuser,
and Thomas Kosch. 2021. Let’s Frets! Mastering Guitar Playing with Capacitive
Sensing and Visual Guidance. In Extended Abstracts of the 2021 CHI Conference on
Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association
for Computing Machinery, New York, NY, USA, Article 169, 4 pages. https:
//doi.org/10.1145/3411763.3451536
[19]
Andrii Matviienko, Wilko Heuten, Alan Dix, and Susanne CJ Boll. 2021. Interac-
tive Technology for Cycling Ideate, Make Remote, Together. In Adjunct Publi-
cation of the 23rd International Conference on Mobile Human-Computer Interaction
(Toulouse & Virtual, France) (MobileHCI ’21). Association for Computing
Machinery, New York, NY, USA, Article 29, 4 pages. https://doi.org/10.1145/
3447527.3474870
[20]
Andrew P. McPherson, Adrian Gierakowski, and Adam M. Stark. 2013. The Space
between the Notes: Adding Expressive Pitch Control to the Piano Keyboard. In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
(Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY,
USA, 2195–2204. https://doi.org/10.1145/2470654.2481302
[21]
Florian Müller, Dirk Schnelle-Walka, Tobias Grosse-Puppendahl, Sebastian Gün-
ther, Markus Funk, Kris Luyten, Oliver Brdiczka, Niloofar Dezfuli, and Max
Mühlhäuser. 2018. SmartObjects: Sixth Workshop on Interacting with Smart Ob-
jects. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Com-
puting Systems (Montreal QC, Canada) (CHI EA ’18). Association for Computing
Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3170427.3170606
[22]
Katja Rogers, Amrei Röhlig, Matthias Weing, Jan Gugenheimer, Bastian Könings,
Melina Klepsch, Florian Schaub, Enrico Rukzio, Tina Seufert, and Michael Weber.
2014. P.I.A.N.O.: Faster Piano Learning with Interactive Projection. In Proceedings
of the Ninth ACM International Conference on Interactive Tabletops and Surfaces
(Dresden, Germany) (ITS ’14). Association for Computing Machinery, New York,
NY, USA, 149–158. https://doi.org/10.1145/2669485.2669514
[23]
Hatice Sahin, Heiko Mueller, Shadan Sadeghian, Debargha Dey, Andreas Löcken,
Andrii Matviienko, Mark Colley, Azra Habibovic, and Philipp Wintersberger. 2021.
Workshop on Prosocial Behavior in Future Mixed Trac. In 13th International
Conference on Automotive User Interfaces and Interactive Vehicular Applications
(Leeds, United Kingdom) (AutomotiveUI ’21 Adjunct). Association for Comput-
ing Machinery, New York, NY, USA, 167–170. https://doi.org/10.1145/3473682.
3477438
[24]
Gian-Luca Savino, Tamara von Sawitzky, Andrii Matviienko, Miriam Sturdee,
Paweł W. Woźniak, Markus Löchtefeld, Andrew L. Kun, Andreas Riener, and
Jonna Häkkilä. 2021. Cycling@CHI: Towards a Research Agenda for HCI in the
Bike Lane. In Extended Abstracts of the 2021 CHI Conference on Human Factors in
Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing
Machinery, New York, NY, USA, Article 107, 5 pages. https://doi.org/10.1145/
3411763.3441316
[25]
Martin Schmitz, Sebastian Günther, Karola Marky, Florian Müller, Andrii Matvi-
ienko, Alexandra Voit, Roberts Marky, Max Mühlhäuser, and Thomas Kosch.
2022. Rethinking Smart Objects: The International Workshop on Interact-
ing with Smart Objects in Interactive Spaces. In Companion Proceedings of the
2022 Conference on Interactive Surfaces and Spaces (Wellington, New Zealand)
(ISS ’22). Association for Computing Machinery, New York, NY, USA, 64–67.
https://doi.org/10.1145/3532104.3571470
[26]
Martin Schmitz, Dominik Schön, Henning Klagemann, and Thomas Kosch. 2024.
3DA: Assessing 3D-Printed Electrodes for Measuring Electrodermal Activity. In
Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing
Systems (Honolulu, HI, USA) (CHI EA ’24). ACM, New York, NY, USA. https:
//doi.org/10.1145/3613905.3650938
[27]
Snehesh Shrestha, Cornelia Fermüller, Tianyu Huang, Pyone Thant Win, Adam
Zukerman, Chethan M Parameshwara, and Yiannis Aloimonos. 2022. AIMu-
sicGuru: Music Assisted Human Pose Correction.
[28]
Snehesh Shrestha, William Sentosatio, Huiashu Peng, Cornelia Fermuller, and
Yiannis Aloimonos. 2023. FEVA: Fast Event Video Annotation Tool.
[29]
John Edel Tamani, Jan Christian Blaise Cruz, Joshua Raphaelle Cruzada, Jolene
Valenzuela, Kevin Gray Chan, and Jordan Aiko Deja. 2018. Building guitar
strum models for an interactive air guitar prototype. In Proceedings of the 4th
International Conference on Human-Computer Interaction and User Experience in
Indonesia, CHIuXiD’18. 18–22.
[30]
Tamara von Sawitzky, Philipp Wintersberger, Andrii Matviienko, Andreas Löcken,
Andreas Riener, and Florian Michahelles. 2021. Workshop on Intelligent Cyclist
Support Systems and Applications. In Mensch und Computer 2021 - Workshop-
band, Carolin Wienrich, Philipp Wintersberger, and Benjamin Weyers (Eds.).
Gesellschaft für Informatik e.V., Bonn, 1–3. https://doi.org/10.18420/muc2021-
mci-ws12- 122
330
ResearchGate has not been able to resolve any citations for this publication.
Conference Paper
Full-text available
The interactive augmentation of musical instruments to foster self-expressiveness and learning has a rich history. Over the past decades, the incorporation of interactive technologies into musical instruments emerged into a new research field requiring strong collaboration between different disciplines. The workshop "Intelligent Music Interfaces" covers a wide range of musical research subjects and directions, including (a) current challenges in musical learning, (b) prototyping for improvements, (c) new means of musical expression, and (d) evaluation of the solutions.
Conference Paper
Full-text available
The increasing proliferation of smart objects in everyday life has changed how we interact with computers. Instead of concentrating computational capabilities and interaction into one device, everyday objects have naturally integrated parts of interactive features. Although this has led to many practical applications, the possibilities for explicit or implicit interaction with such objects are still limited in interaction spaces. We still often rely on smartphones as interactive hubs for controlling smart objects, hence not fulfilling the vision of truly smart objects. The workshop Rethinking Smart Objects invites practitioners and researchers from both academia and industry to discuss novel interaction paradigms and the integration and societal implications of using smart objects in interactive space. This workshop will include an action plan with leading questions, aiming to move the research field forward.
Article
Full-text available
Humans have been developing and playing musical instruments for millennia. With technological advancements, instruments were becoming ever more sophisticated. In recent decades computer-supported innovations have also been introduced in hardware design, usability, and aesthetics. One of the most commonly digitally augmented instruments is the piano. Besides electronic keyboards, several prototypes augmenting pianos with different projections providing various levels of interactivity on and around the keyboard have been implemented in order to support piano players. However, it is still unclear whether these solutions support the learning process. In this paper, we present a systematic review of augmented piano prototypes focusing on instrument learning based on the four themes derived from interviews with piano experts to understand better the problems of teaching the piano. These themes are (i) synchronised movement and body posture, (ii) sight-reading, (iii) ensuring motivation, and (iv) encouraging improvisation. We found that prototypes are saturated on the synchronisation themes, and there are opportunities for sight-reading, motivation, and improvisation themes. We conclude by presenting recommendations on augmenting piano systems towards enriching the piano learning experience as well as on possible directions to expand knowledge in the area.
Conference Paper
Full-text available
The fact that automated vehicles will be part of road traffic raises the question of how human road users, like bicyclists or pedestrians, would safely interact with them. Research has proposed external human-machine interfaces (eHMIs) for automated vehicles as a potential solution. Concept prototypes and evaluations so far have mainly focused on young, healthy adults and people without disabilities, such as visual impairments. For a “one-for-all” holistic, inclusive solution, however, further target groups like children, seniors, or people with (other) special needs will have to be considered. In this workshop, we bring together researchers, experts, and practitioners working on eHMIs to broaden our perspective on inclusiveness. We aim to identify aspects of inclusive eHMI design that can be universal and tailored to any culture and will focus on discussing methods, tools, and scenarios for inclusive communication.
Article
Full-text available
The use of wearable sensor technology opens up exciting avenues for both art and HCI research, providing new ways to explore the invisible link between audience and performer. To be effective, such work requires close collaboration between performers and researchers. In this paper, we report on the co-design process and research insights from our work integrating physiological sensing and live performance. We explore the connection between the audience’s physiological data and their experience during the performance, analyzing a multi-modal dataset collected from 98 audience members. We identify notable moments based on HRV and EDA, and show how the audience’s physiological responses can be linked to the choreography. The longitudinal changes in HRV features suggest a strong connection to the choreographer’s intended narrative arc, while EDA features appear to correspond with short-term audience responses to dramatic moments. We discuss the physiological phenomena and implications for designing feedback systems and interdisciplinary collaborations.
Conference Paper
Full-text available
Prosocial Behavior" means cooperating and acting in a way to benefit others. Since more and more diverse road users (such as electronic bicycles, scooters, etc.) but also vehicles at different levels of automation are sharing the safety-critical road environment, acting prosocial will become increasingly important in the future for both human and automated traffic participants. A few papers so far have already begun to address this issue, but currently, there exist no systematic methodological approaches to research this area. In the proposed workshop, we plan to define more specifically what characterizes prosocial behavior in future traffic scenarios where automated and manual vehicles meet and interact with all kinds of vulnerable road users. We further want to identify important scenarios and discuss potential evaluation methods for researching prosocial behavior. Ultimately, these findings will be integrated into a research agenda actively pursued by cooperation initiated during this event.