Conference PaperPDF Available

Bystander Privacy in Lifelogging

Conference Paper

Bystander Privacy in Lifelogging

Abstract

Lifelogging, a technique for digitally gathering every moment in life using wearable cameras, is a growing phenomenon. However, it is important to remember that in addition to recording their own lives, users are recording others as well, which raises privacy issues around lifelogging from a bystander's perspective. This warrants conducting bystander-focused privacy studies in an uncontrolled environment, by reaping upon the experience of practitioners from multiple domains, but not limited to, urban science, sociology, privacy, usability, psychology, and law, to design such studies.
© The Authors. Proceedings of British
HCI 2016 - Fusion, Bournemouth, UK
Bystander Privacy in Lifelogging
Soumyadeb Chowdhury Md Sadek Ferdous Joemon M Jose
Singapore Institute of Technology University of Southampton University of Glasgow
Soum.Chowdhury@singaporetech.edu.sg S.Ferdous@soton.ac.uk Joemon.Jose@glasgow.ac.uk
Lifelogging, a technique for digitally gathering every moment in life using wearable cameras, is a growing
phenomenon. However, it is important to remember that in addition to recording their own lives, users are
recording others as well, which raises privacy issues around lifelogging from a bystander’s perspective.
This warrants conducting bystander-focused privacy studies in an uncontrolled environment, by reaping
upon the experience of practitioners from multiple domains, but not limited to, urban science, sociology,
privacy, usability, psychology, and law, to design such studies.
Lifelogging, Privacy, Bystander, Experiment Design, Society
1. INTRODUCTION
Lifelogging wearable cameras such as Autographer
(Autographer, 2009) and Narrative Clip (Narrative
clip, 2012) passively capture photos at a very high
rate Autographer (100-200 images per hour) and
Narrative Clip (120 images per hour). These
photographs consist of comprehensive records of
users’ daily activities (Gurrin, 2014). However, it is
important to note that in addition to recording their
own lives, users are recording others as well. This
intrusive and invasive nature of lifelogging has
understandably drawn the attention of advocates of
personal privacy in different domains. The photos
may be captured in different settings/scenarios
including, but not limited to private (living room),
intimate (time spent with loved ones), corporate
(workplace). Hence it is necessary to understand the
privacy implications of the lifelog images from the
perspectives of both the lifeloggers (LLs, i.e. person
wearing the device) and bystanders (BSs i.e. known
or unknown people captured in the photographs).
The use of wearable cameras thus raise serious
questions of permission from the BSs, especially if
the photos are eventually published.
To the best of our knowledge, none of the existing
studies have focused on the privacy of the
bystanders. Some practitioners have suggested that
the privacy of the lifeloggers is more important than
that of bystanders (Lifelogging, 2016). However,
there are no research evidence to support such a
suggestion. The right to privacy is one of the
fundamental human rights in any modern society. It
advocates and facilitates mechanisms to uphold the
privacy of all individuals within the society. However,
what is private is highly debated because there are
social, legal, political, technical and psychological
connotations to privacy (Gurrin et al., 2014).
In this paper, we contend that it is necessary to
conduct longitudinal lifelogging user studies in the
natural environment of the LLs to better understand
the privacy concerns raised by the BSs. Our
contention is based upon the anecdotal evidence
gathered from a study conducted in the United
Kingdom (Glasgow), where a number of BSs (who
were either known or unknown to the LLs) had
raised privacy concerns. This evidence might be
considered a bit conflicting compared to the results
reported in (Hoyle et al., 2014), but it clearly
demonstrates a knowledge gap in the context of
lifelogging privacy, which warrants a systematic
investigation.
2. RELATED STUDIES
From a legal perspective, Allen (2008) has
suggested that lifelogs should not be recorded
without the explicit permission of the bystanders,
and has proposed a counter-technology to block
lifelog surveillance, but did not provide any guidance
on how to implement such a technology. Kelly et al.
(2013) have suggested that privacy and anonymity
of third parties must be protected and image
containing third parties should not be published
without their prior consent. The most recent work
reported in Hoyle et al. (2014 and 2015) have
studied the reactions of the bystanders from a LL’s
perspective. The study demonstrated that 26 out of
36 LLs reported positive interest from the BSs, and
none of the LLs reported to have encountered
opposition from the BSs. We believe that these
results are likely to vary for each study, and may
depend upon the surroundings (when and where a
wearable device is used) and the frequency of use.
For example, there is a difference between using the
device in solitude, among a group of classmates (in
class), group of strangers, or inside a business
Bystander Privacy in Lifelogging, Chowdhury Ferdous Jose
seminar. We also contend that perceptions about
privacy infringements is likely to vary from one
individual to another, individual across different age
groups, and privacy laws enforced in a country.
3. EVALUATION
We recruited 40 LLs (subjects in this case) by
sending emails to the university mailing list, once
ethics approval was granted by the ethics committee
to conduct the study. The LLs were asked to use a
wearable camera, Autographer”, for a period of 3
days (6-8 hours every day). The devices were
handed over for a week, to give some slack time to
the LLs to get used to the devices. LLs were also
provided with an instruction sheet to help them use
the devices. Additionally, a list of suggestions were
provided in order to reduce the potential risks arising
from our study. The suggestions included, but were
not limited to: (1) avoiding using the device in rest
rooms and in places where photography may be
prohibited; (2) if a bystander enquires about the
device, first the image capture must be paused, and
then the objectives of our study must be explained;
(3) if a bystander seems to be concerned, then step
2 should be followed and they should be provided
with our contact information, as well as a URL
containing further details about our project. The LLs
were also asked to note down the date and time of
the incident in a deletion card, so that we were able
to delete the images captured for that period of time.
This framework ensured that the study reflected the
way these devices will be actually used in real-life.
After the study, semi-structured interviews were
conducted with the LLs to understand the reactions
of the bystanders. A majority of the LLs participating
in our study (29 out of 40) had reported that people
they lived with, i.e. either their family members or
flatmates enquired about the device (in particular
about characteristics of the device). A majority (i.e.
around 85%) of the flatmates, and family members
were not in favour of using the device inside the
shared flat. They were concerned about: how the
lifelogs will portray themselves; location where the
contents will be stored; how the contents will be
used; what sort of activities will be captured,
especially when the device is either used in the
private space or homely environment. 18 LLs had
reported that their colleagues in the workplace
enquired about the device, and requested the LLs to
avoid using the device during office hours. There
were 9 instances, where the line manager explicitly
asked the LLs to avoid using the lifelogging cameras
during the office hours, and in the business
gatherings, though there were no policies set by the
organization to prevent people from taking pictures,
during office hours (according to the LLs). It was also
reported that 12 bystanders (unknown people in
public transport, and shopping stores), enquired
about the device, and showed discontent and
concern after hearing its characteristics. The
bystanders politely requested the lifeloggers to
delete their photographs, because they did not like
to the idea of being randomly photographed. These
bystanders also told that they felt as if they were
being tracked. The LLs also pointed out that the
attitude of the people towards privacy seemed to
depend upon: (1) the ability to identify the camera on
the body (i.e. visibility of the device); (2) what sort of
information about the wearable is provided during
the enquiry, and how this information is described.
Thus the experimental protocol used for a study may
also influence the perception of the public. In
summary, the bystanders were concerned about: (1)
self-impression management, how they are being
represented in the lifelogs, (2) being tracked, i.e.
identified being at a certain place engaged in certain
activities; (3) how the lifelogs can be changed to give
a negative impression about them (or activities).
4. CONCLUSION
The recent advances in pervasive sensing,
especially wearable cameras have raised privacy
concerns, especially from the perspectives of the
BSs. We do not claim that our study with LLs is
rigorous, but contend that the results underscore the
need for further studies with BSs to understand their
concerns. But in order to conduct the studies, and
make them comprehensive, and comparable, it is
necessary to establish an evaluation protocol
comprising of a set of common guidelines. This set
of guidelines would need to address issues such as:
consistent way of designing experiments; protocols
to be used, i.e. minimum duration of study, metric to
be reported, interpretation of metrics; type of training
to be given before starting the experiment, where to
place the device on the body; instructions for the LLs
during exigencies. This will help to establish a
common set of criteria that can be used to compare
the results obtained in a study and demonstrate its
viability in a specific context. Practitioners and
experts from multiple domains such as security,
legal, privacy, urban sciences, and psychology,
should come together to establish rules for effective
experimental design to conduct lifelogging privacy
studies, which can be adopted in the future. By
taking such steps, we can: (1) edge closer to design
and development of a privacy-preserving lifelogging
system that protects the privacy as well security of
the bystanders; (2) understand the societal and
urban aspects negatively influencing the acceptance
of such technologies.
ACKNOWLEDGEMENT
The authors acknowledge support from Integrated
Multimedia City Data (iMCD), a project within the
ESRC-funded Urban Big Data Centre
(ES/L011921/1).
Bystander Privacy in Lifelogging, Chowdhury Ferdous Jose
REFERENCES
Allen, A., (2008). Dredging up the past: Lifelogging
memory, and surveillance. The University of
Chicago Law Review, 75(1). 47-74.
Autographer. (2009) The World's First Intelligent
Wearable Camera, OMG Life Ltd Available
from: http://www.autographer.com (Accessed: 20
May 2016).
Gurrin, C., Smeaton, A.F. and Doherty, A.R., (2014).
‘LifeLogging: Personal Big Data. Foundations and
trends in information retrieval, 8(1).1125.
Gurrin, C., Albatal, R., Joho, H. and Ishii, K., (2014).
A privacy by design approach to lifelogging.
Digital Enlightenment Yearbook 2014: Social
Networks and Social Machines, Surveillance and
Empowerment. 49.
Hoyle, R, et al., (2014), September. Privacy
behaviors of lifeloggers using wearable cameras.
In Proceedings of the 2014 ACM International
Joint Conference on Pervasive and Ubiquitous
Computing. 571-582.
Hoyle, R, et al., (2015), April. Sensitive Lifelogs: A
Privacy Analysis of Photos from Wearable
Cameras. In Proceedings of the 33rd Annual
ACM Conference on Human Factors in
Computing Systems. 1645-1648.
Kelly, P, et al., (2013). An ethical framework for
automated, wearable cameras in health behavior
research. American journal of preventive
medicine, 44(3). 314-319.
Lifelogging: The end of privacy - Part 3. (2016).
https://e27.co/lifelogging-the-end-of-privacy-part-
3-20141017/. Accessed: 2016- 03- 18.
Narrative clip 2. (2012). Memoto AB Available at:
http://getnarrative.com (Accessed: 23 May
2016).
... We believe that such innovations can come from the Engineering Interactive Computing Systems community through careful, dedicated consideration of the design and implementation of input/output devices, interaction techniques, user interfaces and, ultimately, complete interactive systems for lifelogging and abstracting life. Moreover, collection of snapshots and videos using wearable video cameras can create opposition from passersby, who may react to the invasion of their privacy [11,34], an aspect that further motivates research towards new, privacy-oriented forms for storing, presenting, and sharing the visual lifelog. ...
... There is a wide literature on smartglasses with topics ranging from technical prototyping [1,7,28] to applications [57,59,60], and to studies and investigations to understand privacy and ethical concerns regarding public video recording with personal, wearable video cameras [11,27,34]. For example, Yang et al. [57] developed a prototype for the visual augmentation of daily life, providing users with a refined visual recognition result computed from multiple input images; Zhao et al. [60] introduced CueSee, a system that enabled people with low vision to locate a specific product on a grocery store shelf independently and efficiently; and the ForeSee system of Zhao et al. [59] was designed to improve the visual experience of users with low vision by providing magnification, contrast and edge enhancement of the visual reality. ...
... For the wearer, inconspicuousness prevents Life-Tags to interfere with everyday life activities and translates to always-on, passive operation. For passersby, it prevents unwanted opposition or manifestation of concerns [11], which should nevertheless be managed appropriately by the wearers of Life-Tags. Previous studies showed that lifeloggers are concerned about and take measures regarding the privacy of bystanders [27]. ...
Article
Full-text available
We introduce "Life-Tags," a wearable, smartglasses-based system for abstracting life in the form of clouds of tags and concepts automatically extracted from snapshots of the visual reality recorded by wearable video cameras. Life-Tags summarizes users' life experiences using word clouds, highlighting the "executive summary" of what the visual experience felt like for the smartglasses user during some period of time, such as a specific day, week, month, or the last hour. In this paper, we focus on (i) design criteria and principles of operation for Life-Tags, such as its first-person, eye-level perspective for recording life, passive logging mode, and privacy-oriented operation, as well as on (ii) technical and engineering aspects for implementing Life-Tags, such as the block architecture diagram highlighting devices, software modules, third-party services, and dataflows. We also conduct a technical evaluation of Life-Tags and report results from a controlled experiment that generated 21,600 full HD snapshots from six indoor and outdoor scenarios, representative of everyday life activities, such as walking, eating, traveling, etc., with a total of 180 minutes of recorded life to abstract with tag clouds. Our experimental results and Life-Tags prototype inform design and engineering of future life abstracting systems based on smartglasses and wearable video cameras to ensure effective generation of rich clouds of concepts, reflective of the visual experience of the smartglasses user.
... They found that the two privacy mechanisms most supported by both the groups are owner registration and automatic face blurring [70]. Besides, research has been conducted on bystanders' privacy concerns in the presence of lifelogging wearable devices both from lifeloggers' and bystanders' points of view [13,27]. ...
... Recent attempts have been made to design effective feedback mechanisms for IoT devices that are more physical in nature. 13 However, these attempts are focused on providing people with visual LED-based feedback to indicate when the device's sensors are actively recording data. LEDs have served as a tangible notion for a device's state for years now. ...
Article
Sensor-enabled computers in the form of 'IoT' devices such as home security cameras and voice assistants are increasingly becoming pervasive in our environment. With the embedded cameras and microphones in these devices, this 'invasion' of our everyday spaces can pose significant threats to the privacy of bystanders. Because of their complex functionality, even when people attempt privacy measures (such as asking the owner to "turn the camera off"), these devices may still record information because of the lack of a 'real' off button. With the ambiguities of current designs, a bystander's perceived privacy can diverge from their actual privacy. Indeed, being able to assess one's actual privacy is a key aspect in managing one's privacy according to Altman's theory of boundary regulation, and current designs fall short in assuring people of their privacy. To understand how people as bystanders manage their privacy with IoT devices, we conducted an interview study about people's perceptions of and behaviors around current IoT devices. We find that although participants' behaviors line up with Altman's theory of boundary regulation, in the face of uncertainty about their privacy, they desire or engage in various 'tangible' workarounds. Based on our findings, we identify and introduce the concept of 'tangible privacy' as being essential to boundary regulation with IoT devices. We argue that IoT devices should be designed in a way that clearly and unambiguously conveys sensor states to people around them and make actionable design recommendations to provide strong privacy assurances to bystanders.
... Privacy has been a primary concern in life-logging since its beginning. Recent studies [25,26] on privacy behaviors of life-loggers have shown that apart from their own sensitive information found in such images, privacy of bystanders captured in these images is also an important factor when they deliberate about sharing the images [8]. It has also been shown that contextual factors, such as location and time, also play a significant part in a users' judgement of the privacy sensitivity of the images [26]. ...
Article
Full-text available
Built-in pervasive cameras have become an integral part of mobile/wearable devices and enabled a wide range of ubiquitous applications with their ability to be "always-on". In particular, life-logging has been identified as a means to enhance the quality of life of older adults by allowing them to reminisce about their own life experiences. However, the sensitive images captured by the cameras threaten individuals' right to have private social lives and raise concerns about privacy and security in the physical world. This threat gets worse when image recognition technologies can link images to people, scenes, and objects, hence, implicitly and unexpectedly reveal more sensitive information such as social connections. In this paper, we first examine life-log images obtained from 54 older adults to extract (a) the artifacts or visual cues, and (b) the context of the image that influences an older life-logger's ability to recall the life events associated with a life-log image. We call these artifacts and contextual cues "stimuli". Using the set of stimuli extracted, we then propose a set of obfuscation strategies that naturally balances the trade-off between reminiscability and privacy (revealing social ties) while selectively obfuscating parts of the images. More specifically, our platform yields privacy-utility tradeoff by compromising, on average, modest 13.4% reminiscability scores while significantly improving privacy guarantees -- around 40% error in cloud estimation.
... Bystander privacy is known to be impacted by pet wearables [68]. Human wearables are no different in this regard, and may indeed go further in how grievous the infringement of bystanders' privacy is depending on the sensors contained in a device and the data collection they enable -microphones recording all ambient audio, video recording what people do in public, and so on [17,31]. We found specific contexts in which users of these wearables impede on bystander privacy. ...
Article
Full-text available
We report on a mixed-method, comparative study investigating whether there is a difference between privacy concerns expressed about pet wearables as opposed to human wearables-and more importantly, why. We extracted the privacy concerns found in product reviews (N=8,038) of pet wearables (activity, location, and dual-function trackers), contrasting the (lack of) concerns and misuse to a curated set of reviews for similar human-oriented wearables (N=20,431). Our findings indicate that, while overall very few privacy concerns are expressed in product reviews, for pet wearables they are expressed even less, even though consumers use these devices in a manner which impacts both personal and bystander privacy. An additional survey of pet owners (N=201) eliciting what factors would cause them to not purchase (or stop using) pet wearables indicated comparably few privacy concerns, strengthening the representativeness of our findings. A thematic analysis reveals that the lack of privacy concerns may be explained by, among other factors, emotional drivers to purchase the device, and prioritization of (desired) functionality to support those emotional drivers over privacy requirements. Moreover, we found that pet wearables are used in different ways than originally intended, which raise novel privacy implications to be dealt with. We propose that in order to move towards more privacy-conscious use of pet wearables, a combination of understanding consumer rationale and behavior as well as ensuring data protection legislation is adequate to real-world use is needed.
Article
Full-text available
We address in this work the topic of broadcasting one’s visual reality, captured by the video cameras embedded in mobile and wearable devices, to a remote audience. We discuss several designs of such life broadcasting systems, which we position at the intersection of lifelogging, Alternate Reality, and Cross-Reality technology. To this end, we introduce the “Alternate Reality Broadcast-Time” matrix for the broadcasting and consumption of alternate realities, in which designs of systems that implement sharing and consumption of personal visual realities can be positioned, characterized, and compared. These design options range from simple video streaming over the web using conventional video protocols to mediated and augmented reality, to audio narration and vibrotactile rendering of concepts automatically detected from video captured by wearable cameras. To demonstrate the usefulness of our broadcast-time matrix, we describe three prototypes implementing lifelogging, concept recognition from video, augmented and mediated vision, and vibrotactile feedback. Our contributions open the way toward new applications that blend lifelogging, consumption of multimedia alternate realities, and XR technology to empower users with richer opportunities for self-expression and new means to connect with the followers of events in their lives.
Article
Intelligent personal assistants (IPA), such as Amazon Alexa and Google Assistant, are becoming increasingly present in multi-user households leading to questions about privacy and consent, particularly for those who do not directly own the device they interact with. When these devices are placed in shared spaces, every visitor and cohabitant becomes an indirect user, potentially leading to discomfort, misuse of services, or unintentional sharing of personal data. To better understand how owners and visitors perceive IPAs, we interviewed 10 in-house users (account owners and cohabitants) and 9 visitors from a student and young professionals sample who have interacted with such devices on various occasions. We find that cohabitants in shared households with regular IPA interactions see themselves as owners of the device, although not having the same controls as the account owner. Further, we determine the existence of a smart speaker etiquette which doubles as trust-based boundary management. Both in-house users and visitors demonstrate similar attitudes and concerns around data use, constant monitoring by the device, and the lack of transparency around device operations. We discuss interviewees' system understanding, concerns, and protection strategies and make recommendation to avoid tensions around shared devices.
Article
Full-text available
Governance in online communities is an increasingly high-stakes challenge, and yet many basic features of offline governance legacies-juries, political parties, term limits, and formal debates, to name a few-are not in the feature-sets of the software most community platforms use. Drawing on the paradigm of Institutional Analysis and Development, this paper proposes a strategy for addressing this lapse by specifying basic features of a generalizable paradigm for online governance called Modular Politics. Whereas classical governance typologies tend to present a choice among wholesale ideologies, such as democracy or oligarchy, Modular Politics would enable platform operators and their users to build bottom-up governance processes from computational components that are modular and composable, highly versatile in their expressiveness, portable from one context to another, and interoperable across platforms. This kind of approach could implement pre-digital governance systems as well as accelerate innovation in uniquely digital techniques. As diverse communities share and connect their components and data, governance could occur through a ubiquitous network layer. To that end, this paper proposes the development of an open standard for networked governance.
Article
A number of wearable 'lifelogging' camera devices have been released recently, allowing consumers to capture images and other sensor data continuously from a first-person perspective. Unlike traditional cameras that are used deliberately and sporadically, lifelogging devices are always 'on' and automatically capturing images. Such features may challenge users' (and bystanders') expectations about privacy and control of image gathering and dissemination. While lifelogging cameras are growing in popularity, little is known about privacy perceptions of these devices or what kinds of privacy challenges they are likely to create. To explore how people manage privacy in the context of lifelogging cameras, as well as which kinds of first-person images people consider 'sensitive,' we conducted an in situ user study (N = 36) in which participants wore a lifelogging device for a week, answered questionnaires about the collected images, and participated in an exit interview. Our findings indicate that: 1) some people may prefer to manage privacy through in situ physical control of image collection in order to avoid later burdensome review of all collected images; 2) a combination of factors including time, location, and the objects and people appearing in the photo determines its 'sensitivity;' and 3) people are concerned about the privacy of bystanders, despite reporting almost no opposition or concerns expressed by bystanders over the course of the study.
Article
We have recently observed a convergence of technologies to foster the emergence of lifelogging as a mainstream activity. Computer storage has become significantly cheaper, and advancements in sensing technology allows for the efficient sensing of personal activities, locations and the environment. This is best seen in the growing popularity of the quantified self movement, in which life activities are tracked using wearable sensors in the hope of better understanding human performance in a variety of tasks. This review aims to provide a comprehensive summary of lifelogging, to cover its research history, current technologies, and applications. Thus far, most of the lifelogging research has focused predominantly on visual lifelogging in order to capture life details of life activities, hence we maintain this focus in this review. However, we also reflect on the challenges lifelogging poses to an information retrieval scientist. This review is a suitable reference for those seeking an information retrieval scientist's perspective on lifelogging and the quantified self.
Article
Technologic advances mean automated, wearable cameras are now feasible for investigating health behaviors in a public health context. This paper attempts to identify and discuss the ethical implications of such research, in relation to existing guidelines for ethical research in traditional visual methodologies. Research using automated, wearable cameras can be very intrusive, generating unprecedented levels of image data, some of it potentially unflattering or unwanted. Participants and third parties they encounter may feel uncomfortable or that their privacy has been affected negatively. This paper attempts to formalize the protection of all according to best ethical principles through the development of an ethical framework. Respect for autonomy, through appropriate approaches to informed consent and adequate privacy and confidentiality controls, allows for ethical research, which has the potential to confer substantial benefits on the field of health behavior research.
The World's First Intelligent Wearable Camera, OMG Life Ltd Available from
  • Autographer
Autographer. (2009) The World's First Intelligent Wearable Camera, OMG Life Ltd Available from: http://www.autographer.com (Accessed: 20 May 2016).
A privacy by design approach to lifelogging
  • C Gurrin
  • R Albatal
  • H Joho
  • K Ishii
Gurrin, C., Albatal, R., Joho, H. and Ishii, K., (2014). A privacy by design approach to lifelogging. Digital Enlightenment Yearbook 2014: Social Networks and Social Machines, Surveillance and Empowerment. 49.
Dredging up the past: Lifelogging memory, and surveillance. The University of Chicago Law Review
  • A Allen
Allen, A., (2008). Dredging up the past: Lifelogging memory, and surveillance. The University of Chicago Law Review, 75(1). 47-74.
Sensitive Lifelogs: A Privacy Analysis of Photos from Wearable Cameras
  • R Hoyle
Hoyle, R, et al., (2015), April. Sensitive Lifelogs: A Privacy Analysis of Photos from Wearable Cameras. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 1645-1648.
The end of privacy -Part 3. (2016). https://e27.co/lifelogging-the-end-of-privacy-part- 3-20141017
  • Lifelogging
Lifelogging: The end of privacy -Part 3. (2016). https://e27.co/lifelogging-the-end-of-privacy-part- 3-20141017/. Accessed: 2016-03-18.
The World's First Intelligent Wearable Camera
  • Autographer
Autographer. (2009) The World's First Intelligent Wearable Camera, OMG Life Ltd Available from: http://www.autographer.com (Accessed: 20 May 2016).