Content uploaded by Shuo Niu
Author content
All content in this area was uploaded by Shuo Niu on Mar 12, 2023
Content may be subject to copyright.
Close-up and Whispering: An Understanding of Multimodal and
Parasocial Interactions in YouTube ASMR videos
ABSTRACT
CCS CONCEPTS
Human-centered computing
→
Empirical studies in collab-
orative and social computing
KEYWORDS
ACM Reference Format:
CHI Conference on
Human Factors in Computing Systems (CHI ’22), April 29-May 5, 2022, New
Orleans, LA, USA.
CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA
1 INTRODUCTION
CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA Shuo Niu, et al.
social aractionphysical araction task
araction
•
•
•
Figure 1: e structure of the research questions
2 BACKGROUND
2.1 ASMR Videos on YouTube
An Understanding of Multimodal and Parasocial Interactions in ASMR videos CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA
2.2 Multimodal Interactions in ASMR videos
visual
soundtouchtaste scenario
Visual interactions
Sound interactions
Touch interactions
Taste interactions
scenario triggers
2.3 Parasocial Attractions in ASMR Videos
“conver-
sational give-and-take,”
“parasocial interactions.”
socialphysical
task
Social araction
Physical araction
CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA Shuo Niu, et al.
Task araction
2.4 ASMR Experiences and Benets
3 ASMR VIDEO DATA
𝑁=9676,4.26%
4 METHODS
4.1 Grounded Analysis
An Understanding of Multimodal and Parasocial Interactions in ASMR videos CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA
Table 1: Subcategories and denition of the ve multimodal
interactions in ASMR videos
Table 2: Subcategories and denition of the three parasocial
attractiveness in ASMR videos
extreme closeupcloseup medium closeup
medium shot medium-full shot
full shot
Table 3: Agreement scores between experts and between ex-
perts and MTurk participants. Calculated using Fleiss’ Kappa
with Jaccard method.
CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA Shuo Niu, et al.
Figure 2: Examples of visual, sound, touch, and scenario subcategories
Figure 3: Closeup, medium, and full shot examples in spatial
proximity
4.2 Video Data Annotation
𝑆𝐷 =654717.5 𝑆𝐷 =9554.74
4.3 Comment Data Processing
𝑆𝐷 =87 .81
An Understanding of Multimodal and Parasocial Interactions in ASMR videos CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA
Table 4: Example words associated with intimacy, relaxation,
and sleep identied by PMI. e word list is used to count
the percentages of words associated with each feeling.
😌
4.4 Statistical Method
𝑎𝑙𝑝ℎ𝑎 =0.05
5 RESULTS
5.1 RQ1: Multimodal Interactions in ASMR
Videos
Visual
Sound
Touch
𝑁=1573,59.07%
Taste
Scenario
5.2 RQ2: Parasocial Attractions and ASMR
Experience Patterns
5.2.1 ASMR for Social Experiences.
𝑣𝑖𝑠𝑢 𝑎𝑙. 𝑓 𝑎𝑐𝑒 𝑡𝑜𝑓 𝑎𝑐𝑒
𝑠𝑜𝑢𝑛𝑑 .𝑤ℎ𝑖𝑠𝑝𝑒𝑟𝑖𝑛𝑔
𝑡𝑜𝑢𝑐ℎ.𝑣𝑖𝑒𝑤 𝑒𝑟
CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA Shuo Niu, et al.
Face
to face
Mukbang
Object only
Images
Serve people
Gaming
Other
Animals
Whispering
Object
Mouth
Body
&Cloth
Mic
Ambience
Other
Objects
Viewer
Own
body
Real person
Taste
None
Service
Fantasy
Romance
Other
0
500
1000
1500
2000
Number
of videos
Visual Sound Touch Taste Scenario
1736
65
.19%
271
10.18% 252
9.46% 229
8.6% 100
3.76% 40
1.5% 22
0.83% 13
0.49%
2080
78.11%
1235
46.38%
843
31.66% 556
20.88% 340
12.77% 179
6.72% 45
1.69%
1003
37.66% 775
29.1%
212
7.96% 107
4.02%
329
12.35%
1908
71.65%
397
14.91% 197
7.4% 118
4.43% 43
1.61%
Figure 4: Le: e distribution of videos in the subcategories of the ve interaction modalities. Right: e co-appearance rates
of visual triggers and triggers of other modalities, calculated using the Jaccard index.
Talk
to
Talk with
None
Gesture
&Text
Closeup
No face
Partial face
Medium
Fullshot
None
Treatment
Eat&Drink
Common
0
400
800
1200
1600
Number of videos
Social Proximity Task
1204
45.21
%
639
24.0% 536
20.13% 284
10.66%
1653
62.07%
550
20.65% 245
9.2% 198
7.44% 17
0.64%
1549
58.17%
578
21.7% 327
12.28% 209
7.85%
Face to face
Mukbang
Object only
Images
Serve people
Whispering
Object
Mouth
Body
&Cloth
Mic
Ambience
Objects
Viewer
Own
body
Real person
Taste
None
Service
Fantasy
Romance
Talk to
Talk with
None
Gesture
&Text
Closeup
No face
Partial face
Medium
None
Treatment
Eat&Drink
Common
Visual Sound Touch Scenario
SocialProximityTask
Figure 5: Le: the distribution of videos in subcategories of parasocial attractions. Right: e signicant associations between
subcategories of multimodal interactions and parasocial attractions. Orange squares are signicant positive associations. Grey
squares are signicant negative associations.
𝑠𝑜𝑐𝑖𝑎𝑙 .𝑡𝑎𝑙𝑘 𝑡𝑜
𝑠𝑜𝑐𝑖𝑎𝑙 .𝑡𝑎𝑙𝑘 𝑤𝑖𝑡 ℎ
𝑡𝑜𝑢𝑐 ℎ.𝑜𝑏 𝑗𝑒 𝑐𝑡𝑠
𝑁=273
𝑁=108
𝑁=81
𝑣𝑖𝑠𝑢𝑎𝑙 .𝑖𝑚𝑎𝑔𝑒𝑠
𝑣𝑖𝑠𝑢𝑎𝑙 .𝑚𝑢𝑘𝑏𝑎𝑛𝑔
𝑣𝑖𝑠𝑢 𝑎𝑙.𝑜𝑏 𝑗 𝑒𝑐𝑡 𝑜𝑛𝑙𝑦
𝑣𝑖𝑠𝑢𝑎𝑙 .𝑠𝑒𝑟 𝑣 𝑒𝑝𝑒𝑜 𝑝𝑙 𝑒
𝑡𝑜𝑢𝑐ℎ.𝑟 𝑒𝑎𝑙 𝑝𝑒𝑟𝑠𝑜𝑛
𝑡𝑎𝑠𝑡𝑒 𝑠𝑐𝑒𝑛𝑎𝑟 𝑖𝑜 .𝑛𝑜𝑛𝑒
5.2.2 ASMR for Intimate Interaction.
𝑣𝑖𝑠𝑢 𝑎𝑙. 𝑓 𝑎𝑐𝑒 𝑡𝑜𝑓 𝑎𝑐𝑒
𝑠𝑜𝑢𝑛𝑑 .𝑤ℎ𝑖𝑠𝑝𝑒𝑟𝑖𝑛𝑔
𝑠𝑜𝑢𝑛𝑑 .𝑜𝑏 𝑗𝑒𝑐 𝑡
𝑠𝑜𝑢𝑛𝑑 .𝑚𝑜𝑢𝑡ℎ
𝑠𝑜𝑢𝑛𝑑 .𝑚𝑖𝑐
𝑡𝑜𝑢𝑐 ℎ.𝑜𝑏 𝑗𝑒 𝑐𝑡𝑠
𝑡𝑜𝑢𝑐ℎ.𝑣𝑖𝑒𝑤 𝑒𝑟
𝑝𝑟𝑜𝑥𝑖𝑚𝑖𝑡𝑦.𝑐𝑙𝑜𝑠𝑒𝑢𝑝
An Understanding of Multimodal and Parasocial Interactions in ASMR videos CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA
(a) e ASMRtist soly whispers
to the viewer.
(b) e ASMRtist talks with
viewer and pretends to reach
viewer’s face in a treatment.
(c) e ASMRtist manipulate a
bottle near the mic while intro-
ducing it in whispers.
(d) e ASMRtist performs a con-
versation in a nerve exam on the
viewer in a treatment roleplay.
(e) e ASMRtist pretends to re-
cruit the viewer into their punk
band in a fantasy roleplay.
(f) e ASMRtist talks with the
viewer as the Goddess of Love in
a romantic, image-only roleplay.
Figure 6: Example ASMR videos showing ASMR interactions
for social attraction.
5.2.3 ASMR for Activity Observation.
𝑁=1549,58.17%
𝑡𝑎𝑠𝑘.𝑛𝑜𝑛𝑒
𝑁=1085
𝑁=210
𝑁=191
(a) An ASMRtist whispers near
the camera/mic to simulate close-
ear speaking.
(b) e ASMRtist manipulates pli-
ers near the camera to generate
tingling sound.
(c) e ASMRtist smacks lips near
the mic to emulate close-ear feel-
ings.
(d) e ASMRtist scratches the
mic and whispers trigger words
(coconut, pickle, etc.).
(e) e ASMRtist pretends to do
makeup on viewer’s face while
the viewer is asleep.
(f) An audio-only ASMR, where
ASMRtist emulates cleaning
viewer’s ears as a sister.
Figure 7: Example ASMR videos showing ASMR interactions
for intimacy.
𝑁=507
𝑁=317
𝑁=164
𝑁=381
𝑡𝑎𝑠𝑘.𝑡𝑟 𝑒𝑎𝑡𝑚𝑒𝑛𝑡
𝑣𝑖𝑠𝑢 𝑎𝑙. 𝑓 𝑎𝑐𝑒 𝑡𝑜𝑓 𝑎𝑐𝑒
𝑁=456
𝑡𝑜𝑢𝑐ℎ.𝑣𝑖𝑒𝑤 𝑒𝑟
𝑁=304
𝑡𝑎𝑠𝑘.𝑡𝑟 𝑒𝑎𝑡𝑚𝑒𝑛𝑡
𝑣𝑖𝑠𝑢𝑎𝑙 .𝑠𝑒𝑟 𝑣 𝑒𝑝𝑒𝑜 𝑝𝑙 𝑒
𝑁=85
𝑡𝑜𝑢𝑐ℎ.𝑟 𝑒𝑎𝑙 𝑝𝑒𝑟𝑠𝑜𝑛
𝑁=83
𝑣𝑖𝑠𝑢𝑎𝑙 .𝑚𝑢𝑘𝑏𝑎𝑛𝑔
𝑁=259
𝑠𝑜𝑢𝑛𝑑 .𝑚𝑜𝑢𝑡ℎ
𝑁=301
𝑠𝑜𝑐𝑖𝑎𝑙 .𝑛𝑜𝑛𝑒
𝑝𝑟𝑜𝑥𝑖𝑚𝑖𝑡𝑦.𝑝𝑎𝑟𝑡𝑖𝑎𝑙𝑓 𝑎𝑐𝑒
𝑇𝑎𝑠𝑘 .𝑐𝑜𝑚𝑚𝑜𝑛
𝑣𝑖𝑠𝑢 𝑎𝑙.𝑜𝑏 𝑗 𝑒𝑐𝑡 𝑜𝑛𝑙𝑦
𝑁=
43
𝑠𝑜𝑢𝑛𝑑 .𝑤ℎ𝑖𝑠𝑝𝑒𝑟𝑖𝑛𝑔
𝑁=184
𝑠𝑜𝑢𝑛𝑑 .𝑜𝑏 𝑗𝑒𝑐 𝑡
𝑁=122
𝑡𝑎𝑠𝑘.𝑐𝑜𝑚𝑚𝑜𝑛
𝑡𝑜𝑢𝑐ℎ.𝑜𝑤 𝑛𝑏𝑜𝑑𝑦
CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA Shuo Niu, et al.
𝑁=38
(a) e ASMRtist scratches the
mic with a brush and makes ran-
dom mouth sound.
(b) An ASMRtist constantly typ-
ing on a keyboard for 1.6 hours
without speaking.
(c) An ASMRtist collapses soaps
and starch without showing
themselves or speaking.
(d) e ASMRtist performs head
massage on another person in the
video.
(e) e ASMRtist consumes a
large quantity of food.
(f) e ASMRtist eats cakes and
jelly and explains with captions.
(g) e ASMRtist shows magic
tricks with cards.
(h) e ASMRtist put makeup on
face while chewing gum.
Figure 8: Example ASMR videos showing no-task and with-
task ASMR interactions.
5.3
RQ3: Viewers’ Comments about the Feelings
of ASMR Experience
Emotion Tone
0
20
40
60
80
100
Score
(100 is positive, 0 is negative)
ASMR Expressive writing Natural speech Twitter
Social words Body words Perception words
0
2
4
6
8
10
% of total words
Figure 9: Comparison of emotion tone and percentages of
social, body, and perception words between ASMR videos
and expressive writing, natural speech, and Twitter [51]
5.3.1 Social and Intimacy.
𝑠𝑜𝑐𝑖𝑎𝑙 .𝑡𝑎𝑙𝑘 𝑡𝑜
𝑠𝑜𝑐𝑖𝑎𝑙 .𝑡𝑎𝑙𝑘 𝑤𝑖𝑡 ℎ
𝑠𝑜𝑐𝑖𝑎𝑙 .𝑔𝑒𝑠𝑡𝑢𝑟𝑒&𝑡𝑒𝑥 𝑡
𝑠𝑜𝑐𝑖𝑎𝑙 .𝑛𝑜𝑛𝑒
𝑝𝑟𝑜𝑥𝑖𝑚𝑖𝑡𝑦.𝑐𝑙𝑜𝑠𝑒𝑢𝑝
𝑝𝑟𝑜𝑥𝑖𝑚𝑖𝑡𝑦.𝑚𝑒𝑑𝑖𝑢𝑚
An Understanding of Multimodal and Parasocial Interactions in ASMR videos CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA
Table 5: Results of multivariate LSR models that predict the percentages of feeling words in comments. Only the p-values of
signicant predictors are presented. Adjusted
𝛼=0.0083
. *Multi-choice factors,
𝑝
is from the dummy variable with the smallest
p-value.
𝐹 𝑝 𝑟2𝑝𝑠𝑜𝑐𝑖 𝑎𝑙 𝑝𝑝ℎ𝑦𝑠𝑖 𝑐𝑎𝑙 𝑝𝑡𝑎𝑠 𝑘 𝐹 𝑝 𝑟 2𝑝𝑣𝑖𝑠𝑢𝑎𝑙 𝑝𝑠𝑜𝑢 𝑛𝑑 𝑝𝑡𝑜𝑢𝑐ℎ 𝑝𝑡 𝑎𝑠𝑡𝑒 𝑝𝑠 𝑐𝑒𝑛𝑎𝑟 𝑖𝑜
Talk with
Talk to
Gesture
&Text
None
0
5
10
15
20
Social
words
(%)
Social
*** ***
***
***
***
Closeup
Medium
No
face
Partial face
Proximity
*** ***
***
***
***
Images
Face to face
Serve people
Mukbang
Object only
Visual
****
***
*** ***
***
Whispering
Ambience
Body
&Cloth
Object
Mouth
Mic
Sound
********* **
Talk with
Talk to
Gesture
&Text
None
0
.0
0.5
1.0
1.5
2.0
2.5
Intimate
words (%)
Social
******
***
***
***
No face
Medium
Closeup
Partial face
Proximity
***
***
***
Treatment
Common
None
Eat&Drink
Task
***
***
***
***
Images
Serve people
Face to face
Object only
Mukbang
Visual
**** ***
*** *** ***
***
******
Ambience
Whispering
Body&Cloth
Object
Mic
Mouth
Sound
** *********
Fantasy
Romance
Service
None
Scenario
***
**
***
***
Figure 10: Social and intimate word frequencies between subcategories of signicant multimodal and parasocial predictors.
Ordered by the average percentage in descending order. Horizontal bars show signicant dierences (
𝑝
* < 0.05,
𝑝
** < 0.01,
𝑝
*** <
0.001).
𝑝𝑟𝑜𝑥𝑖𝑚𝑖𝑡𝑦.𝑛𝑜𝑓 𝑎𝑐𝑒
𝑣𝑖𝑠𝑢𝑎𝑙 .𝑖𝑚𝑎𝑔𝑒𝑠
𝑠𝑜𝑢𝑛𝑑 .𝑎𝑚𝑏𝑖𝑒𝑛𝑐𝑒
5.3.2 Body and Perception.
𝑡𝑎𝑠𝑘.𝑒𝑎𝑡 &𝑑𝑟𝑖𝑛𝑘
𝑠𝑜𝑐𝑖𝑎𝑙 .𝑔𝑒𝑠𝑡𝑢𝑟𝑒&𝑡𝑒𝑥 𝑡
CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA Shuo Niu, et al.
𝑝𝑟𝑜𝑥𝑖𝑚𝑖𝑡𝑦.𝑝𝑎𝑟𝑡𝑖𝑎𝑙𝑓 𝑎𝑐𝑒
𝑡𝑎𝑠𝑘.𝑛𝑜𝑛𝑒
𝑠𝑐𝑒𝑛𝑎𝑟𝑖𝑜.𝑛𝑜𝑛𝑒
𝑣𝑖𝑠𝑢 𝑎𝑙.𝑜𝑏 𝑗 𝑒𝑐𝑡 𝑜𝑛𝑙𝑦
𝑠𝑜𝑢𝑛𝑑 .𝑜𝑏 𝑗𝑒𝑐 𝑡
𝑠𝑜𝑢𝑛𝑑 .𝑚𝑖𝑐
5.3.3 Relaxation and Sleepiness.
𝑡𝑎𝑠𝑘.𝑡𝑟 𝑒𝑎𝑡𝑚𝑒𝑛𝑡
𝑠𝑐𝑒𝑛𝑎𝑟𝑖𝑜.𝑠𝑒𝑟 𝑣𝑖𝑐𝑒
𝑣𝑖𝑠𝑢𝑎𝑙 .𝑠𝑒𝑟 𝑣 𝑒𝑝𝑒𝑜 𝑝𝑙 𝑒
𝑡𝑜𝑢𝑐ℎ.𝑟 𝑒𝑎𝑙 𝑝𝑒𝑜𝑝𝑙𝑒
𝑡𝑜𝑢𝑐ℎ.𝑣𝑖𝑒𝑤 𝑒𝑟
𝑠𝑜𝑢𝑛𝑑 .𝑚𝑖𝑐
6 DISCUSSION
6.1 ASMR as an Experience of Multimodal
Social Connection
6.2 ASMR as an Experience of Relaxing
Physical Intimacy
An Understanding of Multimodal and Parasocial Interactions in ASMR videos CHI ’22, April 29-May 5, 2022, New Orleans, LA, USA
Gesture&Text
None
Talk to
Talk with
0
10
20
30
40
50
Body
words (%)
Social
*****
***
***
***
Partial face
Medium
Closeup
No face
Proximity
*** ***
***
***
***
Eat&Drink
None
Common
Treatment
Ta