No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Exploring Lifelog Sharing and Privacy
Abstract
The emphasis on exhaustive passive capturing of images using wearable cameras like Autographer, which is often known as lifelogging has brought into foreground the challenge of preserving privacy, in addition to presenting the vast amount of images in a meaningful way. In this paper, we present a user-study to understand the importance of an array of factors that are likely to influence the lifeloggers to share their lifelog images in their online circle. The findings are a step forward in the emerging area intersecting HCI, and privacy, to help in exploring design directions for privacy mediating techniques in lifelogging applications.
... Interestingly, we do not always share captured data publicly but just store it on a personal storage [33] (some statistics are shown in Appendix A), and such private data contains various moments of everyday life that are not found in publicly shared data [4,12,32,36]. Accessing exten-sive amounts of private data could lead to tremendous advances in computer vision and AI technology. ...
... Specifically, let Θ be the RHS of Equation (12), which is known when R(·) is identified and γ t , λ is estimated. Then, when using some specific loss functions we can solve ∇w t (z t ,w t ) = Θ for z t as follows: ...
We propose a privacy-preserving framework for learning visual classifiers by leveraging distributed private image data. This framework is designed to aggregate multiple classifiers updated locally using private data and to ensure that no private information about the data is exposed during its learning procedure. We utilize a homomorphic cryptosystem that can aggregate the local classifiers while they are encrypted and thus kept secret. To overcome the high computational cost of homomorphic encryption of high-dimensional classifiers, we (1) impose sparsity constraints on local classifier updates and (2) propose a novel efficient encryption scheme named doubly-permuted homomorphic encryption (DPHE) which is tailored to sparse high-dimensional data. DPHE (i) decomposes sparse data into its constituent non-zero values and their corresponding support indices, (ii) applies homomorphic encryption only to the non-zero values, and (iii) employs double permutations on the support indices to make them secret. Our experimental evaluation on several public datasets demonstrates that the proposed approach significantly outperforms other privacy-preserving methods and achieves comparable performance against state-of-the-art visual recognition methods without privacy preservation.
... As supervised tasks require every single training sample to be annotated with its ground truth label, an earlier work proposed leveraging crowdsourcing platforms to mitigate such high annotation costs [1]. However, as the training data are assumed to be made public, this approach is not applicable for certain data that the owners wish to keep private, such as life-logging videos [2], biological data [3], and medical data [4]. ...
This paper addresses the problem of decentralized learning to achieve a high-performance global model by asking a group of clients to share local models pre-trained with their own data resources. We are particularly interested in a specific case where both the client model architectures and data distributions are diverse, which makes it nontrivial to adopt conventional approaches such as Federated Learning and network co-distillation. To this end, we propose a new decentralized learning method called Decentralized Learning via Adaptive Distillation (DLAD). Given a collection of client models and a large number of unlabeled distillation samples, the proposed DLAD 1) aggregates the outputs of the client models while adaptively emphasizing those with higher confidence in given distillation samples and 2) trains the global model to imitate the aggregated outputs. Our extensive experimental evaluation on multiple public datasets (MNIST, CIFAR-10, and CINIC-10) demonstrates the effectiveness of the proposed method.
... Their results were partly confirmed by Price et al., who, however, found no significant differences in sharing when a screen was present [Price et al. 2017]. Chowdhury et al. found that whether lifelogging imagery is suitable for sharing is (in addition to content, scenario, and location) mainly determined by its sensitivity [Chowdhury et al. 2016]. Ferdous et al. proposed a set of guidelines that, among others, include semi-automatic procedures to determine the sensitivity of captured images according to user-provided preferences [Ferdous et al. 2017]. ...
Eyewear devices, such as augmented reality displays, increasingly integrate eye tracking, but the first-person camera required to map a user's gaze to the visual scene can pose a significant threat to user and bystander privacy. We present PrivacEye, a method to detect privacy-sensitive everyday situations and automatically enable and disable the eye tracker's first-person camera using a mechanical shutter. To close the shutter in privacy-sensitive situations, the method uses a deep representation of the first-person video combined with rich features that encode users' eye movements. To open the shutter without visual input, PrivacEye detects changes in users' eye movements alone to gauge changes in the "privacy level" of the current situation. We evaluate our method on a first-person video dataset recorded in daily life situations of 17 participants, annotated by themselves for privacy sensitivity, and show that our method is effective in preserving privacy in this challenging setting.
... generate diverse samples that can be observed collectively under various client's environments, and will ultimately benefit many applications including image translation [4,5,6], anomaly detection [7], data compression [8], and domain adaptation [9]. However, curation of client's private data should be prohibited due to privacy concerns (e.g., life-logging videos [10], biological data [11], and medical data [12]). This dilemma between data utility and privacy makes it hard to aggregate all the client data in a central server, necessitating decentralized learning approaches. ...
This work addresses a new problem of learning generative adversarial networks (GANs) from multiple data collections that are each i) owned separately and privately by different clients and ii) drawn from a non-identical distribution that comprises different classes. Given such multi-client and non-iid data as input, we aim to achieve a distribution involving all the classes input data can belong to, while keeping the data decentralized and private in each client storage. Our key contribution to this end is a new decentralized approach for learning GANs from non-iid data called Forgiver-First Update (F2U), which a) asks clients to train an individual discriminator with their own data and b) updates a generator to fool the most `forgiving' discriminators who deem generated samples as the most real. Our theoretical analysis proves that this updating strategy indeed allows the decentralized GAN to learn a generator's distribution with all the input classes as its global optimum based on f-divergence minimization. Moreover, we propose a relaxed version of F2U called Forgiver-First Aggregation (F2A), which adaptively aggregates the discriminators while emphasizing forgiving ones to perform well in practice. Our empirical evaluations with image generation tasks demonstrated the effectiveness of our approach over state-of-the-art decentralized learning methods.
... Their results were partly confirmed by Price et al. who, however, found no significant differences in sharing when a screen was present [38]. Chowdhury et al. [10] found that whether lifelogging imagery is suitable for sharing is (in addition to content, scenario, and location) mainly determined by its sensitivity. Ferdous et al. proposed a set of guidelines that, amongst others, include semi-automatic procedures to determine the sensitivity of captured images according to userprovided preferences [14]. ...
As first-person cameras in head-mounted displays become increasingly prevalent, so does the problem of infringing user and bystander privacy. To address this challenge, we present PrivacEye, a proof-of-concept system that detects privacysensitive everyday situations and automatically enables and disables the first-person camera using a mechanical shutter. To close the shutter, PrivacEye detects sensitive situations from first-person camera videos using an end-to-end deep-learning model. To open the shutter without visual input, PrivacEye uses a separate, smaller eye camera to detect changes in users' eye movements to gauge changes in the "privacy level" of the current situation. We evaluate PrivacEye on a dataset of first-person videos recorded in the daily life of 17 participants that they annotated with privacy sensitivity levels. We discuss the strengths and weaknesses of our proof-of-concept system based on a quantitative technical evaluation as well as qualitative insights from semi-structured interviews.
... Interestingly, we do not always share captured data pub- licly but just store it on a personal storage [32], and such pri- vate data contains various moments of everyday life that are not found in publicly shared data [5,11,31,33]. Accessing extensive amounts of private data could lead to tremendous advances in computer vision and AI technology. ...
Objective: Hand function impairment after cervical spinal cord injury (SCI) can significantly reduce independence. Unlike current hand function assessments, wearable camera systems could potentially measure functional hand usage at home, and thus benefit the development of neurorehabilitation strategies. The objective of this study was to understand the views of individuals with SCI on the use of wearable cameras to track neurorehabilitation progress and outcomes in the community.
Design: Questionnaires.
Setting: Home simulation laboratory.
Participants: Fifteen individuals with cervical SCI.
Outcome Measures: After using wearable cameras in the simulated home environment, participants completed custom questionnaires, comprising open-ended and structured questions.
Results: Participants showed relatively low concerns related to data confidentiality when first-person videos are used by clinicians (1.93 ± 1.28 on a 5-point Likert scale) or researchers (2.00 ± 1.31). Storing only automatically extracted metrics reduced privacy concerns. Though participants reported moderate privacy concerns (2.53 ± 1.51) about wearing a camera in daily life due to certain sensitive situations (e.g. washrooms), they felt that information about their hand usage at home is useful for researchers (4.73 ± 0.59), clinicians (4.47 ± 0.83), and themselves (4.40 ± 0.83). Participants found the system moderately comfortable (3.27 ± 1.44), but expressed low desire to use it frequently (2.87 ± 1.36).
Conclusion: Despite some privacy and comfort concerns, participants believed that the information obtained would be useful. With appropriate strategies to minimize the data stored and recording duration, wearable cameras can be a well-accepted tool to track function in the home and community after SCI.
The Microsoft SenseCam is a small lightweight wearable camera used to passively capture photos and other sensor readings from a user's day-to-day activities. It can capture up to 3,000 images per day, equating to almost 1 million images per year. It is used to aid memory by creating a personal multimedia lifelog, or visual recording of the wearer's life. However the sheer volume of image data captured within a visual lifelog creates a number of challenges, particularly for locating relevant content. Within this work, we explore the applicability of semantic concept detection, a method often used within video retrieval, on the novel domain of visual lifelogs. A concept detector models the correspondence between low-level visual features and high-level semantic concepts (such as indoors, outdoors, people, buildings, etc.) using supervised machine learning. By doing so it determines the probability of a concept's presence. We apply detection of 27 everyday semantic concepts on a lifelog collection composed of 257,518 SenseCam images from 5 users. The results were then evaluated on a subset of 95,907 images, to determine the precision for detection of each semantic concept. We conduct further analysis on the temporal consistency, co-occurance and trends within the detected concepts to more extensively investigate the robustness of the detectors within this novel domain. We additionally present future applications of concept detection within the domain of lifelogging.
While media reports about wearable cameras have focused on the privacy concerns of bystanders, the perspectives of the `lifeloggers' themselves have not been adequately studied. We report on additional analysis of our previous in-situ lifelogging study in which 36 participants wore a camera for a week and then reviewed the images to specify privacy and sharing preferences. In this Note, we analyze the photos themselves, seeking to understand what makes a photo private, what participants said about their images, and what we can learn about privacy in this new and very different context where photos are captured automatically by one's wearable camera. We find that these devices record many moments that may not be captured by traditional (deliberate) photography, with camera owners concerned about impression management and protecting private information of both themselves and bystanders.
A number of wearable 'lifelogging' camera devices have been released recently, allowing consumers to capture images and other sensor data continuously from a first-person perspective. Unlike traditional cameras that are used deliberately and sporadically, lifelogging devices are always 'on' and automatically capturing images. Such features may challenge users' (and bystanders') expectations about privacy and control of image gathering and dissemination. While lifelogging cameras are growing in popularity, little is known about privacy perceptions of these devices or what kinds of privacy challenges they are likely to create.
To explore how people manage privacy in the context of lifelogging cameras, as well as which kinds of first-person images people consider 'sensitive,' we conducted an in situ user study (N = 36) in which participants wore a lifelogging device for a week, answered questionnaires about the collected images, and participated in an exit interview. Our findings indicate that: 1) some people may prefer to manage privacy through in situ physical control of image collection in order to avoid later burdensome review of all collected images; 2) a combination of factors including time, location, and the objects and people appearing in the photo determines its 'sensitivity;' and 3) people are concerned about the privacy of bystanders, despite reporting almost no opposition or concerns expressed by bystanders over the course of the study.
We present a novel dataset and novel algorithms for the problem of detecting activities of daily living (ADL) in firstperson camera views. We have collected a dataset of 1 million frames of dozens of people performing unscripted, everyday activities. The dataset is annotated with activities, object tracks, hand positions, and interaction events. ADLs differ from typical actions in that they can involve long-scale temporal structure (making tea can take a few minutes) and complex object interactions (a fridge looks different when its door is open). We develop novel representations including (1) temporal pyramids, which generalize the well-known spatial pyramid to approximate temporal correspondence when scoring a model and (2) composite object models that exploit the fact that objects look different when being interacted with. We perform an extensive empirical evaluation and demonstrate that our novel representations produce a two-fold improvement over traditional approaches. Our analysis suggests that real-world ADL recognition is “all about the objects,” and in particular, “all about the objects being interacted with.”
In this paper, we propose a computational model of the recognition of real world scenes that bypasses the segmentation and the processing of individual objects or regions. The procedure is based on a very low dimensional representation of the scene, that we term the Spatial Envelope. We propose a set of perceptual dimensions (naturalness, openness, roughness, expansion, ruggedness) that represent the dominant spatial structure of a scene. Then, we show that these dimensions may be reliably estimated using spectral and coarsely localized information. The model generates a multidimensional space in which scenes sharing membership in semantic categories (e.g., streets, highways, coasts) are projected closed together. The performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category.
The SenseCam is a wearable camera that passively captures approximately 3,000 images per day, which equates to almost one million images per year. It is used to create a personal visual recording of the wearer's life and generates information which can be helpful as a human memory aid. For such a large amount of visual information to be of any use, it is accepted that it should be structured into "events", of which there are about 8,000 in a wearer's average year. In automatically segmenting SenseCam images into events, it will then be useful for users to locate other events similar to a given event e.g. "what other times was I walking in the park?", "show me other events when I was in a restaurant". On two datasets of 240k and 1.8M images containing topics with a variety of information needs, we evaluate the fusion of MPEG-7, SIFT, and SURF content-based retrieval techniques to address the event search issue. We have found that our proposed fusion approach of MPEG-7 and SURF offers an improvement on using either of those sources or SIFT individually, and we have also shown how a lifelog event is modeled has a large effect on the retrieval performance.
The SenseCam is a passively capturing wearable camera, worn around the neck and takes an average of almost 2,000 images per day, which equates to over 650,000 images per year.It is used to create a personal lifelog or visual recording of the wearer's life and generates information which can be helpful as a human memory aid. For such a large amount of visual information to be of any use, it is accepted that it should be structured into "events", of which there are about 8,000 in a wearer's average year. In automatically segmenting SenseCam images into events, it is desirable to automatically emphasise more important events and decrease the emphasis on mundane/routine events. This paper introduces the concept of novelty to help determine the importance of events in a lifelog. By combining novelty with face-to-face conversation detection, our system improves on previous approaches. In our experiments we use a large set of lifelog images, a total of 288,479 images collected by 6 users over a time period of one month each.
A privacy by design approach to lifelogging
- C Gurrin
Gurrin, C, et al., 2014. A privacy by design
approach to lifelogging. Digital Enlightenment
Yearbook 2014: Social Networks and Social
Machines, Surveillance and Empowerment.
Combining image descriptors to effectively retrieve events from visual lifelogs
- A R Doherty
Doherty, A.R, et al., 2008, October. Combining
image descriptors to effectively retrieve events
from visual lifelogs. In Proceedings of the 1st
ACM international conference on Multimedia
information retrieval (pp. 10-17). ACM.
Engineering privacy by design.Computers, Privacy & Data Protection
- S Gürses
- C Troncoso
- C Diaz
Gürses, S., Troncoso, C. and Diaz, C., 2011.
Engineering privacy by design.Computers,
Privacy & Data Protection, 14, p.3.
Sensitive Lifelogs: A Privacy Analysis of Photos from Wearable Cameras
- R Hoyle
Hoyle, R, et al., 2015, April. Sensitive Lifelogs:
A Privacy Analysis of Photos from Wearable
Cameras. In Proceedings of the 33rd Annual
ACM Conference on Human Factors in
Computing Systems (pp. 1645-1648). ACM.
A survey on life logging data capturing
- L M Zhou
- C Gurrin
Zhou, L.M. and Gurrin, C., 2012. A survey on
life logging data capturing.SenseCam 2012.
Engineering privacy by design
- Gürses S.