Content uploaded by Julia Niemann
Author content
All content in this area was uploaded by Julia Niemann on Apr 10, 2015
Content may be subject to copyright.
Julia Niemann & Michael Schenk
Online Experience Sampling Method
Measuring User Behavior Combining Tracking and Survey Methodology
General Online Research 2014, March 5
th
-7
th
, Cologne
Contact
julia.niemann@uni-hohenheim.de | michael.schenk@uni-hohenheim.de
Forschungsstelle für Medienwirtschaft und Kommunikationsforschung
Acknowledgement
The study presented on this poster was part of a research project funded by
Landesmedienanstalt Nordrhein-Westfalen (LfM), Germany.
Measuring User Behavior: Online Experience Sampling Method (ESM)
Online ESM
Online behavior can be measured cost-efficiently
via tracking without causing overly discomfort for
the participants. Nonetheless, such methods of
observation are limited, if user behavior is to be
explained or if media effects are of interest. Thus,
implementing online-surveys into tracking-studies
will provide additional meaning to mere observa-
tion (Online ESM). Direct advantages of Online ESM
are obvious: The interrogation takes place immedi-
ately after a user showed a given behavior a and
without disruption in media use.
Experience Sampling Method (ESM)
Ex-post-surveys demand a considerable amount
of cognitive effort and eventually might suffer
from memory lacks and answering errors. Hence,
the goal of ESM is to evaluate behavior, experi-
ences and attitudes directly within every-day
situations. For that purpose, a series of self-
observations has to be initialized: Participants are
repeatedly activated to fill out a short ques-
tionnaire.
Hektner, J. M., Schmidt, J. A. & Csikszentmihalyi, M. (2007). Experi-
ence sampling method: Measuring the quality of everyda y life.
Thousand Oaks: Sage.
Technical Implementation of Online ESM
Participants install a browser plug-in. The
plugin redirects the traffic through a server.
When a participating client executes a pre-
defined action (e.g. to send a web-form) the
server displays the short ESM survey. Partici-
pants fill out the questionnaire immediately
Client Server Website
plug-in
Case-Study: Adolescents´ and Young Adults´ Self-Disclosure on Social-Networking Sites
Research Problem: Privacy-Paradox
Although the users of the social web care deeply
about privacy issues, they share a lot of data
online.
Discrepancy between attitude and behavior.
Method & Data
A) Survey among 1301 adolescents and young
adults using Social Networking Sites (SNS),
quoted by age, sex and formal education.
Recruitment via online access panel.
B) Followed by a three week tracking and Online
ESM with a subsample (n = 171)
» When participants performed a pre-defined
privacy-relevant action on one of four SNS
(e. g. to add friends, to post a status update)
a short-questionnaire was delivered, based
on a random generator. In total 1479 ques-
tionnaires were filled out during the Online
ESM.
» In the ESM questionnaire participants rated
on a 5-point scale to what degree they…
… thought of disadvantages and conse-
quences for their privacy (risks)
… thought of advantages the action will
have (benefits)
» Participants were invited to reflect on their
experience after the tracking
period in a follow-up survey:
Results
The results of the conventional online survey confirmed the inconsistency of the privacy paradox: There
is only a minor correlation between frequency of online self-disclosure and privacy concern (Pearson´s r
= .07; p < .05; n = 1301). Online ESM provided further insights to the privacy paradox:
„Sometimes it was annoying to fill out a
questionnaire after every status update.“
Added Value of Online ESM:
» Participants appreciated the study. They found it interesting
and motivating. Only marginal signs of reactiveness.
» No media discontinuity between situation and survey.
» Sampling based on actual behavior.
» Control over the responses.
» Decrease of response errors compared to conventional ESM
3.13
2.73
2.64
2.77
2.39
2.7
2.5
2.19
2.26
2.38
1
2
3
4
5
To change information
in the user profile
(n = 34)
To post a
status update
(n = 98)
To post on
someones wall
(n = 96)
To add new
friends
(n = 85)
To confirm
new friends
(n = 71)
T-tests for matched samples, M (SD)
benefits risks
Because of the multilevel-structure event means were centered on participant-level for the analysis.
t(70) = 0,01
n. s.
t(84) = 4.29
p <.001
t(95) = 3.93
p <.001
t(88) = 2.43
p <.05
t(33) = 2.3
p <.05
(1.26)
(1.36)
(1.34)
(1.28)
(1.30)
(1.20)
(1.27)
(1.19)
(1.30) (1.36)
Conclusion
Online ESM revealed that – at least for most of the regarded privacy relevant actions – self-disclosure
on SNS is indeed rather driven by (affective) benefits then by perceived privacy risks.
„ This study showed me how very recklessly I
treat my own data on Facebook. Thank you
for shaking me up :).“
„Thank you for this
marvelous study.
Participating was fun!“
„I simply forgot that the
program was installed :)“
