Conference Paper

A User-Study Examining Visualization of Lifelogs

Conference Paper

A User-Study Examining Visualization of Lifelogs

If you want to read the PDF, try requesting it from the authors.

Abstract

With continuous advances in the pervasive sensing and lifelogging technologies for the quantified self, users now can record their daily life activities automatically and seamlessly. In the existing lifelogging research, visualization techniques for presenting the lifelogs and evaluating the effectiveness of such techniques from a lifelogger's perspective has not been adequately studied. In this paper, we investigate the effectiveness of four distinct visualization techniques for exploring the lifelogs, which were collected by 22 lifeloggers who volunteered to use a wearable camera and a GPS device simultaneously, for a period of 3 days. Based on a user study with these 22 lifeloggers, which required them to browse through their personal lifelogs, we seek to identify the most effective visualization technique. Our results suggest various ways to augment and improve the visualization of personal lifelogs to enrich the quality of user experience and making lifelogging tools more engaging. We also propose a new visualization feature-drill-down approach with details-on-demand, to make the lifelogging visualization process more meaningful and informative to the lifeloggers.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... In [27], the output of Tong et al.'s algorithm was considered the best performing feature that can be used for image quality assessment using neural networks. Also, Tong et al.'s algorithm can be used to filter out blurred images in many applications, for example the lifelogging wearable cameras [28]. ...
... Hence, the compression and encryption must be done on-the-fly before transmitting the images to the cloud. Also, since a lot of the captured images may suffer from blur, most probably they will be deleted later on by the life-loggers [28]. Instead of deleting the blurred images at a later stage, a better option is to filter out the images directly after they are captured by the camera. ...
... There is no solution presented in the literature, software, or hardware, that can perform the image blur detection in parallel with compression and encryption. In most systems, the blur detection is usually performed at a later stage after transmitting the compressed images to the cloud [28]. ...
Article
Full-text available
This paper presents a 3 in 1 standalone FPGA system which can perform color image blur detection in parallel with compression and encryption. Both blur detection and compression are based on the 3-level Haar wavelet transform, which is used as a common building block to save the resources. The compression is based on performing the hard thresholding scheme followed by the Run Length Encoding (RLE) technique. The encryption is based on the 128-bit Advanced Encryption Standard (AES), which is considered one of the most secure algorithms. Moreover, the modified Lorenz chaotic system is combined with the AES to perform the Cipher Block Chaining (CBC) mode. The proposed system is realized using HDL and implemented using Xilinx on XC5VLX50T FPGA. The system has utilized only 25% of the available slices. Furthermore, the system can achieve a throughput of 3.458 Gbps, which is suitable for real-time applications. To validate the compression performance, the system has been tested with all the standard 256×256 images. It is shown that depending on the amount of details in the image, the system can achieve 30dB PSNR at compression ratios in the range of (0.08-0.38). The proposed system can be integrated with digital cameras to process the captured images on-the-fly prior to transmission or storage. Based on the application, the blurred images can be either marked for future enhancement or simply filtered out.
... In relation to photos captured with Lifelogging cameras, Yang and Gurrin [11] discuss different options, including a visual diary and calendar summary view. Chowdhury et al. [1] also study different visualisations including a map-based one. Further, more abstract options are discussed by Duane et al. [2]. ...
... Further, more abstract options are discussed by Duane et al. [2]. While map-based visualisations are often mentioned as useful in certain contexts and also used by some (e.g., [1,6]), most of the time, people see such visualisations of lifelog data as an end in itself. That is, the pure representation of the information is the sole purpose of it. ...
Conference Paper
Full-text available
We present a virtual reality system for accessing geotagged photos taken with a lifelogging camera. Photos are spatially located on a world map that can be explored with a head-mounted display. Using a virtual reality headset allows users to easily and intuitively explore this large information space. Images are initially represented by icons but become visible once a user gets closer to a particular area of interest. While not suitable for all search tasks, this visualisation has benefits in situations where location plays a significant role; be it because the actual content is location-related or because the owner of the lifelog remembers and associates the related event with certain places. Likewise, our spatial representation of the data often implicitly reveals a temporal relationship, which can be helpful in the search process as well.
... generating key frames for daily life review. A recent study of his team [2] investigated the e ectiveness of four distinct techniques used for memory reminiscence. Four distinct techniques represent for (a) visual-temporal; (b) visual-spatial; (c) visual-temporal-spatial; (d) trending location. ...
... In general, we try to investigate the powerful and meaningful methods for assisting autobiographical memory. With the insight of prior work [2,11], we propose two approaches utilizing the biophysical data (heart rate) and lifelogs (images & GPS) for generating the e ective visual retrievable memory. Two approaches, called Special Moment approach and Spatial Frequency approach, are applied in aiding user's daily life reviewing. ...
Article
Full-text available
In this paper, we investigate the effectiveness of two distinct techniques (Special Moment Approach & Spatial Frequency Approach) for reviewing the lifelogs, which were collected by lifeloggers who were willing to use a wearable camera and a bracelet simultaneously for two days. Generally, Special moment approach is a technique for extracting episodic events and Spatial frequency approach is a technique for associating visual with temporal and location information, especially heat map is applied as the spatial data for expressing frequency awareness. Based on that, the participants were asked to fill in two post-study questionnaires for evaluating the effectiveness of those two techniques and their combination. The preliminary result showed the positive potential of exploring individual lifelogs using our approaches.
... Specifically in relation to standalone retrieval efforts, early research on lifelog retrieval has focused on using images as unit of retrieval (e.g. Lee et al. 2008) with some early work in supporting user browsing these image collections (Doherty et al. 2011), or on the use of maps metadata, such as GPS locations, to organise content visually (Chowdhury et al. 2016). Once again, we refer the reader to (Gurrin et al. 2014b) for an overview of early efforts at lifelog search and retrieval. ...
Book
Full-text available
This open access book summarizes the first two decades of the NII Testbeds and Community for Information access Research (NTCIR). NTCIR is a series of evaluation forums run by a global team of researchers and hosted by the National Institute of Informatics (NII), Japan. The book is unique in that it discusses not just what was done at NTCIR, but also how it was done and the impact it has achieved. For example, in some chapters the reader sees the early seeds of what eventually grew to be the search engines that provide access to content on the World Wide Web, today’s smartphones that can tailor what they show to the needs of their owners, and the smart speakers that enrich our lives at home and on the move. We also get glimpses into how new search engines can be built for mathematical formulae, or for the digital record of a lived human life. Key to the success of the NTCIR endeavor was early recognition that information access research is an empirical discipline and that evaluation therefore lay at the core of the enterprise. Evaluation is thus at the heart of each chapter in this book. They show, for example, how the recognition that some documents are more important than others has shaped thinking about evaluation design. The thirty-three contributors to this volume speak for the many hundreds of researchers from dozens of countries around the world who together shaped NTCIR as organizers and participants. This book is suitable for researchers, practitioners, and students—anyone who wants to learn about past and present evaluation efforts in information retrieval, information access, and natural language processing, as well as those who want to participate in an evaluation task or even to design and organize one.
... Specifically in relation to standalone retrieval efforts, early research on lifelog retrieval has focused on using images as unit of retrieval (e.g. Lee et al. 2008) with some early work in supporting user browsing these image collections (Doherty et al. 2011), or on the use of maps metadata, such as GPS locations, to organise content visually (Chowdhury et al. 2016). Once again, we refer the reader to (Gurrin et al. 2014b) for an overview of early efforts at lifelog search and retrieval. ...
Chapter
Full-text available
Lifelogging can be described as the process by which individuals use various software and hardware devices to gather large archives of multimodal personal data from multiple sources and store them in a personal data archive, called a lifelog. The Lifelog task at NTCIR was a comparative benchmarking exercise with the aim of encouraging research into the organisation and retrieval of data from multimodal lifelogs. The Lifelog task ran for over 4 years from NTCIR-12 until NTCIR-14 (2015.02–2019.06); it supported participants to submit to five subtasks, each tackling a different challenge related to lifelog retrieval. In this chapter, a motivation is given for the Lifelog task and a review of progress since NTCIR-12 is presented. Finally, the lessons learned and challenges within the domain of lifelog retrieval are presented.
... [18][19][20] Such technologies span from social media apps such as Facebook that persuades its active user base to regularly upload pictures and share personal information 20 to mobile apps designed to persuade people to lead more active lifestyles. 9,14,17,20,44 The Persuasive System Design (PSD) model expands the earlier work of Fogg 20 and provides a framework to assist in the design of persuasive systems 34 . The PSD model is composed of four persuasive categorical elements, namely (a) Primary Task Support, (b) Dialogue Support, (c) System Credibility Support, and (d) Social Support. ...
Article
Self-tracking feedback with engaging and persuasive visualizations not only helps convey data but can also affect people's attitudes and behaviors. We investigate persuasive self-tracking feedback by augmenting data videos (DVs)-novel, engaging storytelling media. We introduce a new class of DVs, called Persuasive Data Videos (PDVs), by incorporating four persuasive elements-primary task, dialogue, system credibility, and social supports-drawn from the Persuasive System Design Model. We describe the iterative design of PDVs and a within-subjects preliminary validation to check their persuasive potential. We then assess PDVs' feasibility using the Persuasive Potential Questionnaire in a between-subjects study comparing a PDV against a conventional DV on Amazon Mechanical Turk (N = 252). Our results indicate the feasibility of using PDVs in providing individuals' self-tracking feedback to convey persuasive health messages, based on which we discuss opportunities for designing persuasive behavioral feedback in an engaging way.
... According to the user-study done by [Cho16], visualization should be insightful, intuitive, interactive, impressive and immersive. Also, the studied lifeloggers prefer a visualization based on the most frequently visited locations in visually appealing and informative interface that another based on temporal clustering, tempo-spatial clustering or tempo-spatial-visual clustering. ...
Conference Paper
Full-text available
Nowadays, taking photos and recording our life are daily task for the majority of people. The recorded information helped to build several applications like the self-monitoring of activities, memory assistance and long-term assisted living. This trend, called lifelogging, interests a lot of research communities such as computer vision, machine learning, human-computer interaction, pervasive computing and multimedia. Great effort have been made in the acquisition and the storage of captured data but there are still challenges in managing, analyzing, indexing, retrieving, summarizing and visualizing these captured data. In this work, we present a new model driven architecture for deep learning-based multimodal lifelog retrieval, summarization and visualization. Our proposed approach is based on different models integrated in an architecture established on four phases. Based on Convolutional Neural Network, the first phase consists of data preprocessing for discarding noisy images. In a second step, we extract several features to enhance the data description. Then, we generate a semantic segmentation to limit the search area in order to better control the runtime and the complexity. The second phase consist in analyzing the query. The third phase which based on Relational Network aims at retrieving the data matching the query. The final phase treat the diversity-based summarization with k-means which offers, to lifelogger, a key-frame concept and context selection-based visualization.
... For more vague, exploratory search needs, the uniqueness of lifelog data requires us to think about new, innovative ways to visualize and interactively explore them. Researchers are now applying information visualization techniques to lifelog data, including data captured automatically by smartphones [8] and lifelog images [1], but these approaches focus more on passive representation and often lack in possibilities for interactive exploration. An interesting new development is this context is the usage of new platforms for access, such as Virtual and Augmented Reality [3]. ...
Conference Paper
Lifelogging is becoming an increasingly important topic of research and this paper highlights the thoughts of the three panelists at the LSC - Lifelog Search Challenge at ICMR 2018 in Yokohama, Japan on June 11, 2018. The thoughts cover important topics such as the need for challenges in multimedia access, the need for a better user interface and the challenges in building datasets and organising benchmarking activities such as the LSC.
... For example, the GUI can display the lifelogs based on spatio-temporal attributes or advanced visualisation mechanisms (e.g. clustering based on visual or event similarities) [12,13]. ...
Article
The visual lifelogging activity enables a user, the lifelogger, to passively capture images from a first-person perspective and ultimately create a visual diary encoding every possible aspect of her life with unprecedented details. In recent years, it has gained popularities among different groups of users. However, the possibility of ubiquitous presence of lifelogging devices specifically in private spheres has raised serious concerns with respect to personal privacy. In this article, we have presented a thorough discussion of privacy with respect to visual lifelogging. We have readjusted the existing definition of lifelogging to reflect different aspects of privacy and introduced a first-ever privacy threat model identifying several threats with respect to visual lifelogging. We have also shown how the existing privacy guidelines and approaches are inadequate to mitigate the identified threats. Finally, we have outlined a set of requirements and guidelines that can be used to mitigate the identified threats while designing and developing a privacy-preserving framework for visual lifelogging.
Chapter
Lifelog can be described as a digital library of an individual’s life, which is known for its ability to record life and help with memory. Autographer, a wearable camera that can captured images automatically, is always used for aiding episodic memory in lifelog system. In order to improve the effectiveness of retrieving memory using lifelog, this paper proposed two novelty user-relative memory cues to extract important memories for lifeloggers. They are special sentiment cue and special movement cue. With the integration of 2 Autographers and sensors embedded in Android smartphone, we implement a web-based lifelog viewer for lifeloggers to conveniently retrieve memories. On account of our system, we invited some participants to test the usability and efficiency of using our system. The preliminary result showed positive potential of aiding episodic memory by using our approaches.
Chapter
In this paper, we investigate the effectiveness of two distinct techniques (Special Moment Approach and Spatial Frequency Approach) for reviewing the lifelogs, which were collected using a wearable camera and a bracelet, simultaneously for two days. Special moment approach is a technique for extracting episodic events. Spatial frequency approach is a technique for associating visual with temporal and location information. Heat map is applied as the spatial data for expressing frequency awareness. Based on this, the participants were asked to fill in two post-study questionnaires for evaluating the effectiveness of those two techniques and their combination. The preliminary result showed the positive potential of exploring individual lifelogs using our approaches.
Conference Paper
Full-text available
Lifelogging devices, which seamlessly gather various data about a user as they go about their daily life, have resulted in users amassing large collections of noisy photographs (e.g. visual duplicates, image blur), which are difficult to navigate, especially if they want to review their day in photographs. Social media websites, such as Facebook, have faced a similar information overload problem for which a number of summarization methods have been proposed (e.g. news story clustering, comment ranking etc.). In particular, Facebook's Year in Review received much user interest where the objective for the model was to identify key moments in a user's year, offering an automatic visual summary based on their uploaded content. In this paper, we follow this notion by automatically creating a review of a user's day using lifelogging images. Specifically, we address the quality issues faced by the photographs taken on lifelogging devices and attempt to create visual summaries by promoting visual and temporal-spatial diversity in the top ranks. Conducting two crowdsourced evaluations based on 9k images, we show the merits of combining time, location and visual appearance for summarization purposes.
Conference Paper
Full-text available
With the prevalence of digital cameras, the number of digital images increases quickly, which raises the demand for image quality assessment in terms of blur. Based on the edge type and sharpness analysis, using the Harr wavelet transform, a new blur detection scheme is proposed in this paper, which can determine whether an image is blurred or not and to what extent an image is blurred. Experimental results demonstrate the effectiveness of the proposed scheme.
Chapter
A variety of life-tracking devices are being created to give opportunity to track our daily lives accurately and automatically through the application of sensing technologies. Technology allows us to automatically and passively record life activities in previously unimaginable detail, in a process called lifelogging. Captured materials may include text, photos/video, audio, location, Bluetooth logs and information from many other sensing modalities, all captured automatically by wearable sensors. Experience suggests that it can be overwhelming and impractical to manually scan through the full contents of these lifelogs. A promising approach is to apply visualization to large-scale data-driven lifelogs as a means of abstracting and summarizing information. In this chapter, we outline various UI templates that support different visualization schemes.
Article
We have recently observed a convergence of technologies to foster the emergence of lifelogging as a mainstream activity. Computer storage has become significantly cheaper, and advancements in sensing technology allows for the efficient sensing of personal activities, locations and the environment. This is best seen in the growing popularity of the quantified self movement, in which life activities are tracked using wearable sensors in the hope of better understanding human performance in a variety of tasks. This review aims to provide a comprehensive summary of lifelogging, to cover its research history, current technologies, and applications. Thus far, most of the lifelogging research has focused predominantly on visual lifelogging in order to capture life details of life activities, hence we maintain this focus in this review. However, we also reflect on the challenges lifelogging poses to an information retrieval scientist. This review is a suitable reference for those seeking an information retrieval scientist's perspective on lifelogging and the quantified self.
Conference Paper
Of all lifelogging tools, activity trackers are probably among the most widely used ones receiving most public attention. However, when used on a long-term basis e.g. for prevention and wellbeing, the devices’ acceptance by the user and its usability become critical issues. In a user study we explored how activity trackers are used and experienced in daily life. We identified critical issues with regard not just to the HCI topics wearability, appearance of the device, and display and interaction, but also to aspects of modeling and describing the measured and presented data. We suggest four guidelines for the design of future activity trackers. Ideally, activity tracking would be fulfilled by a modular concept of building blocks for sensing, interaction and feedback that the user can freely combine, distribute and wear according to personal preferences and situations.
Article
The growing availability of data in electronic form, the expansion of the World Wide Web (WWW) and the accessibility of computational methods for large-scale data processing have allowed researchers in Information Retrieval (IR) to design systems which can effectively and efficiently constrain search within the boundaries given by context, thus transforming classical search into contextual search. Because of the constraints imposed by context, contextual search better focuses on the user's relevance and improves retrieval performance, since the out-of-context aspects of the search carried out by users that are likely linked to irrelevant documents are left apart. This survey introduces contextual search within a computational framework based on contextual variables, contextual factors and statistical models. The framework adopted in this survey considers the data observable from the real world entities participating in contextual search and classifies them as whatwe call contextual variables. The contextual variables considered are content, geotemporal, interaction, and social variables. Moreover, we distinguish between contextual variables and contextual factor: the former is what can be observed, the latter is what cannot be observed, yet this is the factor affecting the user's relevance assessment. Therefore, in this survey, we describe how statistical models can process contextual variables to infer the contextual factors underlying the current search context. In this survey we provide a background to the subject by: placing it among other surveys on relevance, interaction, context, and behavior; providing the description of the contextual variables used for implementing the statistical models which represent and predict relevance and contextual factors; citing and surveying useful publications to the reader for further examination; providing an overview of the evaluation methodologies and findings relevant to this subject; and briefly describing some implementations of contextual search tools.
Conference Paper
A number of researchers have been building high-level semantic concept detectors such as outdoors, face, building, etc., t o help with semantic video retrieval. Using the TRECVID video collection and LSCOM truth annotations from 300 concepts, we simulate perfor- mance of video retrieval under different assumptions of concept detection accuracy. Even low detection accuracy provides good re- trieval results, when sufficiently many concepts are used. C onsid- ering this extrapolation under reasonable assumptions, this paper arrives at the conclusion that "concept-based" video retri eval with fewer than 5000 concepts, detected with minimal accuracy of 10% mean average precision is likely to provide high accuracy results, comparable to text retrieval on the web, in a typical broadcast news collection. We also derive evidence that it should be feasible to find sufficiently many new, useful concepts that would be help ful for retrieval.
Article
Extremely high correlations between repeated judgments of visual appeal of homepages shown for 50 milliseconds have been interpreted as evidence for a mere exposure effect [Lindgaard et al. 2006]. Continuing that work, the present research had two objectives. First, it investigated the relationship between judgments differing in cognitive demands. Second, it began to identify specific visual attributes that appear to contribute to different judgments. Three experiments are reported. All used the stimuli and viewing time as before. Using a paradigm known to disrupt processing beyond the stimulus offset, Experiment 1 was designed to ensure that the previous findings could not be attributed to such continued processing. Adopting a within-subject design, Experiment 2 investigated the extent to which judgments differing in cognitive demands (visual appeal, perceived usability, trustworthiness) may be driven by the visual characteristics of a Web page. It also enabled analyses of visual attributes that contributed most to the different judgments. Experiment 3 replicated Experiment 2 but using a between-subject design to ensure that no practice effect could occur. The results suggest that all three types of judgments are largely driven by visual appeal, but that cognitively demanding judgments are processed in a qualitatively different manner than visual appeal, and that they rely on somewhat different visual attributes. A model accounting for the results is provided.
Article
This paper describes recent research in subjective usability measurement at IBM. The focus of the research was the application of psychometric methods to the development and evaluation of questionnaires that measure user satisfaction with system usability. The primary goals of this paper are to (1) discuss the psychometric characteristics of four IBM questionnaires that measure user satisfaction with computer system usability, and (2) provide the questionnaires, with administration and scoring instructions. Usability practitioners can use these questionnaires with confidence to help them measure users' satisfaction with the usability of computer systems.
Article
Designing the User Interface is intended primarily for designers, managers, and evaluators of interactive systems. It presents a broad survey of designing, implementing, managing, maintaining, training, and refining the user interface of interactive systems. The book's second audience is researchers in human-computer interaction, specifically those who are interested in human performance with interactive systems. These researchers may have backgrounds in computer science, psychology, information systems, library science, business, education, human factors, ergonomics, or industrial engineering; all share a desire to understand the complex interaction of people with machines. Students in these fields also will benefit from the contents of this book. It is my hope that this book will stimulate the introduction of courses on user-interface design in all these and other disciplines. Finally, serious users of interactive systems will find that the book gives them a more thorough understanding of the design questions for user interfaces. My goals are to encourage greater attention to the user interface and to help develop a rigorous science of user-interface design.
Article
Usability testing is a dynamic process that can be used throughout the process of developing interactive multimedia software. The purpose of usability testing is to find problems and make recommendations to improve the utility of a product during its design and development. For developing effective interactive multimedia software, dimensions of usability testing were classified into the general categories of: learnability; performance effectiveness; flexibility; error tolerance and system integrity; and user satisfaction. In the process of usability testing, evaluation experts consider the nature of users and tasks, tradeoffs supported by the iterative design paradigm, and real world constraints to effectively evaluate and improve interactive multimedia software. Different methods address different purposes and involve a combination of user and usability testing, however, usability practitioners follow the seven general procedures of usability testing for effective multimedia development. As the knowledge about usability testing grows, evaluation experts will be able to choose more effective and efficient methods and techniques that are appropriate to their goals.
Article
Humans can recognize the gist of a novel image in a single glance, independent of its complexity. How is this remarkable feat accomplished? On the basis of behavioral and computational evidence, this paper describes a formal approach to the representation and the mechanism of scene gist understanding, based on scene-centered, rather than object-centered primitives. We show that the structure of a scene image can be estimated by the mean of global image features, providing a statistical summary of the spatial layout properties (Spatial Envelope representation) of the scene. Global features are based on configurations of spatial scales and are estimated without invoking segmentation or grouping operations. The scene-centered approach is not an alternative to local image analysis but would serve as a feed-forward and parallel pathway of visual processing, able to quickly constrain local feature analysis and enhance object recognition in cluttered natural scenes.
Article
A useful starting point for designing advanced graphical user interfaces is the Visual InformationSeeking Mantra: Overview first, zoom and filter, then details-on-demand. But this is only a starting point in trying to understand the rich and varied set of information visualizations that have been proposed in recent years. This paper offers a task by data type taxonomy with seven data types (1-, 2-, 3-dimensional data, temporal and multi-dimensional data, and tree and network data) and seven tasks (overview, zoom, filter, details-on-demand, relate, history, and extract). The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations Ben Shneiderman Department of Computer Science, Human-Computer Interaction Laboratory, and Institute for Systems Research University of Maryland College Park, Maryland 20742 USA ben@cs.umd.edu Abstract: A useful starting point for designing advanced graphical user interfaces is the Visual Information-Seeking Mantra: Overview first, ...
Fitbit official site for activity Trackers & more
  • Fitbit
Fitbit, "Fitbit official site for activity Trackers & more," [Online].
Free Geocoding API Available: https://www.geocode.farm. Accessed: Feb
  • Geocode Farm
Geocode Farm, "Free Geocoding API,". [Online]. Available: https://www.geocode.farm. Accessed: Feb. 14, 2016.
The world's First intelligent Wearable camera. Clip-on, go out
  • Autographer
Autographer, "The world's First intelligent Wearable camera. Clip-on, go out," Autographer. [Online]. Available: http://www.autographer.com. Accessed: Feb. 14, 2016.
Calculate distance and bearing between two Latitude/Longitude points using Haversine formula in JavaScript
  • Chris Veness
Chris Veness "Calculate distance and bearing between two Latitude/Longitude points using Haversine formula in JavaScript." Movable Type Scripts (2011).
Designing the user interface. Pearson Education India
  • Ben Shneiderman
Ben Shneiderman. Designing the user interface. Pearson Education India, 2003.