Figure 10 - uploaded by Vaibhav Pandey
Content may be subject to copyright.
Heat map of bio-variables and summary scores that affect each individual subject. This type of visualization integrates cross-modal data in a manner that a clinician, hospital, public health agency, or any expert can use to monitor health of a patient panel. Clicking on a certain box would pull up further insights and details.
Source publication
Individuals create and consume more diverse data about themselves today than any time in history. Sources of this data include wearable devices, images, social media, geospatial information and more. A tremendous opportunity rests within cross-modal data analysis that leverages existing domain knowledge methods to understand and guide human health....
Contexts in source publication
Context 1
... placing various sensors on the engine, engineers and pilots are able to monitor the status of an engine in real-time and understand when to take precaution or perform an action to ensure the safety and longevity of the engine. We present a similar view of health data in Figure 9 for individual use and Figure 10 for professional/expert use. ...
Context 2
... the approach described in section 4, we assimilate various biological parameters for each of the 24 subjects as shown in Figure 10. We find that even though most of these subjects are all cycling athletes, they have a wide range in both their bio-variables and environmental exposures. ...
Context 3
... step further would be to link this with their electronic medical record system. Looking at this data panel across the subject pool, we discover some interesting trends in Figure 10. As expected, as the age increases the overall heart health state decreases, since age is a large factor of cardiac health (age range in panel is 18- 57). ...
Context 4
... data assimilation and visualizations have been used extensively to maintain the health state of jet engines and other mechanical devices [35]. By placing various sensors on the engine, engineers and pilots are able to monitor the status of an engine in real-time and understand when to take precaution or perform an action to ensure the safety and longevity of the engine. We present a similar view of health data in Figure 9 for individual use and Figure 10 for professional/expert ...
Context 5
... the approach described in section 4, we assimilate various biological parameters for each of the 24 subjects as shown in Figure 10. We find that even though most of these subjects are all cycling athletes, they have a wide range in both their bio-variables and environmental exposures. Current day primary care doctors would not be able to see this when a patient ...
Context 6
... data assimilation can be used inform personal health state as shown in Figure 9. The combination of sensors, IoT devices, and environmental data connections can provide a rich experience to interact with meaningful health insights. This can be for individual use, or for use in when a user visits a health care provider. One step further would be to link this with their electronic medical record system. Looking at this data panel across the subject pool, we discover some interesting trends in Figure 10. As expected, as the age increases the overall heart health state decreases, since age is a large factor of cardiac health (age range in panel is 18- 57). As age increases, we also see a reduction in crime and noise pollution, suggesting that older individuals live in safer and quieter neighborhoods. We also see circadian rhythm disruption maximally in the middle ages (20)(21)(22)(23)(24)(25)(26)(27)(28)(29), suggested a more erratic lifestyle for those in their twenties. Circulation and metabolism scores also trend (including VO2 Max / CRF) lower as age increases. Although this is a small sample size to make any strong conclusions, we can begin to see the power of using this cross data analysis for ...
Citations
... Chow et al. [40] constructed a personalized physiological state monitoring platform using a Bayesian generative model that can be trained for each user independently. Nag et al. [41] built a quantitative state estimation system on cardiovascular health by fusing open source data streams from multiple users and established biomedical domain knowledge. Later, Nag et al. [12] tested the feasibility of personalized optimization through experiments with the cardiovascular personalized optimization system, whose experiments showed that the personalized health optimization can be a powerful adjuvant or alternative therapy for the prevention of chronic cardiovascular disease. ...
Driving activities occupy more and more time for moderns and often can elicit bad states like stress, fatigue, or anger, which can significantly impact road safety and driver health. Therefore, health issues caused by driving should be taken seriously. Whichever the combination of bad health states, it may lead to serious consequences during driving, as evidenced by the large number of traffic accidents that occur each year due to various health issues. As a result of rapid advances in multimedia and sensor technologies, driver health can be automatically detected using multimodal measurements. Therefore, a system that includes driver health detection and health navigation is needed to continuously monitor driver health states and navigate drivers to positive health states to ensure safe driving. In this article, we survey recent related works on driver health detection, as well as discuss some of the main challenges and promising areas to stimulate progress in personalized health navigation for drivers. Finally, we propose a cybernetic-based personalized health navigation framework for drivers (PHN-D), which provides a new paradigm in the field of driver health.
... As shown in figure 1, it (PFM) has two main components: 1) the biological component and 2) the preferential component. The biological component determines how different food items interacts with our biology and health [23,20]. In contrast, the preferential component captures how different contextual and environmental factors impact our food preferences and, in turn, affect our choices. ...
... Some studies have explored the causal aspect modelling in other health related applications such as health state estimation [23] and context-driven nutrition recommendation [22]. The eventual goal of causal aspect modelling, especially for food events, is to enable navigation context-driven health state navigation [21] that would help individuals achieve their health-related goals. ...
A personal food model (PFM) is essential for high-quality food recommendation systems to enhance health and enjoyment. We can build such models using food logging platforms that capture the users’ food events. As proposed in the Westermann and Jain event model, capturing six facets of multi-modal data provides a holistic view of any event. Five of these facets are captured during the event (temporal, structural, informational, experiential, spatial), while the sixth facet is related to the causality of the event. This causal facet is needed to build a robust PFM if all the other relevant information in the aforementioned five facets are captured. Any food logger and subsequent processing should collect all this data in the food event. Ultimately, we want to know what caused this person to eat this food and what changes this food event causes in the person’s health state. In this paper, we identify details of the food event model that may help build a causal understanding in PFM to address the first aspect of the causality, what may be the contextual factors that cause a certain food event to occur for a user. We utilize an event mining approach to determine the causal relationships to build a contextual understanding of the PFM. We generate data using a food event simulator that can generate needed food event data for a person with known PFM. The event mining results uncover this hidden PFM and demonstrate the greater efficacy of this approach than a traditionally designed PFM.
... The food we perceptually enjoy is a complete multimedia experience [27], [51], which extends further to an extensive multimodal effect in the body, impacting the physiology and biochemistry of the individual. A multitude of sensors can measure the relationship between foods and the individual through the dynamic health state variables [34]. These include readily available sensors that provide continuous data collection for blood glucose, heart rate, perspiration rate, and body temperature [19]. ...
... From affecting mood and demeanor to chronic conditions such as cardiovascular disease and diabetes, sleep is an integral part of leading a healthy and happy life [3,4]. The quality of sleep affects the health state of all individuals [12,16]. Although individuals spend a third of their lives sleeping, there are still many questions surrounding sleep quality and how it affects human health. ...
... We can harness all the IoT devices, wearables, and smart devices an individual interacts with to create a multi-modal data set that captures different aspects of a person's sleep and lifestyle. The relevant data streams can be used to create a personalized crossmodal sleep model [16]. This estimation will be personalized based on user data and maintain adaptability through daily updates. ...
... We found that a heart rate strap recorded to a wrist watch (Suunto Spartan Sport), provided acceptable agreement for heart rate utilizing thresholds normally applied to laboratory-based research. The Suunto Spartan Sport device has been evaluated with respect to step count accuracy [36], and proposed as a wearable capable of returning the cardiorespiratory fitness component of an integrated cross-modal cybernetic health status assessment [37]. The device has also been utilized in an outdoor environment to track altitude profile during a 64 km ultra-endurance race [38] and Grand Canyon rim to rim hike [39], and pacing and stride variations during a 44 km trail run performed in tropical conditions [40]. ...
Validation of heart rate responses in wearable technology devices is generally composed of laboratory-based protocols that are steady state in nature and as a result, high accuracy measures are returned. However, there is a need to understand device validity in applied settings that include varied intensities of exercise. The purpose was to determine concurrent heart rate validity during trail running. Twenty-one healthy participants volunteered (female n = 10, [mean (SD)]: age = 31 [11] years, height = 173.0 [7] cm, mass = 75.6 [13] kg). Participants were outfitted with wearable technology devices (Garmin Fenix 5 wristwatch, Jabra Elite Sport earbuds, Motiv ring, Scosche Rhythm+ forearm band, Suunto Spartan Sport watch with accompanying chest strap) and completed a self-paced 3.22 km trail run while concurrently wearing a criterion heart rate strap (Polar H7 heart rate monitor). The trail runs were out-and-back with the first 1.61 km in an uphill direction, and the 1.61 return being downhill in nature. Validity was determined through three methods: Mean Absolute Percent Error (MAPE), Bland-Altman Limits of Agreement (LOA), and Lin’s Concordance Coefficient (rC). Validity measures overall are as follows: Garmin Fenix 5 (MAPE = 13%, LOA = -32 to 162, rC = 0.32), Jabra Elite Sport (MAPE = 23%, LOA = -464 to 503, rC = 0.38), Motiv ring (MAPE = 16%, LOA = -52 to 96, rC = 0.29), Scosche Rhythm+ (MAPE = 6%, LOA = -114 to 120, rC = 0.79), Suunto Spartan Sport (MAPE = 2%, LOA = -62 to 61, rC = 0.96). All photoplethysmography-based (PPG) devices displayed poor heart rate agreement during variable intensity trail running. Until technological advances occur in PPG-based devices allowing for acceptable agreement, heart rate in outdoor environments should be obtained using an ECG-based chest strap that can be connected to a wristwatch or other comparable receiver.
... The food we perceptually enjoy is a complete multimedia experience [27], [51], which extends further to an extensive multimodal effect in the body, impacting the physiology and biochemistry of the individual. A multitude of sensors can measure the relationship between foods and the individual through the dynamic health state variables [34]. These include readily available sensors that provide continuous data collection for blood glucose, heart rate, perspiration rate, and body temperature [19]. ...
Food is central to life. Food provides us with energy and foundational building blocks for our body and is also a major source of joy and new experiences. A significant part of the overall economy is related to food. Food science, distribution, processing, and consumption have been addressed by different communities using silos of computational approaches. In this paper, we adopt a person-centric multimedia and multimodal perspective on food computing and show how multimedia and food computing are synergistic and complementary. Enjoying food is a truly multimedia experience involving sight, taste, smell, and even sound, that can be captured using a multimedia food logger. The biological response to food can be captured using multimodal data streams using available wearable devices. Central to this approach is the Personal Food Model. Personal Food Model is the digitized representation of the food-related characteristics of an individual. It is designed to be used in food recommendation systems to provide eating-related recommendations that improve the user's quality of life. To model the food-related characteristics of each person, it is essential to capture their food-related enjoyment using a Preferential Personal Food Model and their biological response to food using their Biological Personal Food Model. Inspired by the power of 3-dimensional color models for visual processing, we introduce a 6-dimensional taste-space for capturing culinary characteristics as well as personal preferences. We use event mining approaches to relate food with other life and biological events to build a predictive model that could also be used effectively in emerging food recommendation systems.
... Since these tests are so invasive, they cannot be performed frequently enough to have a constant understanding of the health of an individual. This highlights the need for using unobtrusively collected lifestyle data for estimating individual health parameters [35]. ...
Knowing the state of our health at every moment in time is critical for advances in health science. Using data obtained outside an episodic clinical setting is the first step towards building a continuous health estimation system. In this paper, we explore a system that allows users to combine events and data streams from different sources to retrieve complex biological events, such as cardiovascular volume overload. These complex events, which have been explored in biomedical literature and which we call interface events, have a direct causal impact on relevant biological systems. They are the interface through which the lifestyle events influence our health. We retrieve the interface events from existing events and data streams by encoding domain knowledge using an event operator language.
... Significant efforts were spent in studying the relations between images and various subjective attributes, such as sentiment [5,19,46], aesthetics [7,8,21,28,30,41], wellness [10,35], memorability [20,38] and interestingness [11]. Most of these works follow the paradigm of obtaining labels of the subjective attributes at the image level and adopt the same annotation-dependant methodologies that are commonly used in the case of objective attributes [4,33]. ...
Recent years have seen unprecedented research on using artificial intelligence to understand the subjective attributes of images and videos. These attributes are not objective properties of the content but are highly dependent on the perception of the viewers. Subjective attributes are extremely valuable in many applications where images are tailored to the needs of a large group, which consists of many individuals with inherently different ideas and preferences. For instance, marketing experts choose images to establish specific associations in the consumers' minds, while psychologists look for pictures with adequate emotions for therapy. Unfortunately, most of the existing frameworks either focus on objective attributes or rely on large scale datasets of annotated images, making them costly and unable to clearly measure multiple interpretations of a single input. Meanwhile, we can see that users or organizations often interact with images in a multitude of real-life applications, such as the sharing of photographs by brands on social media or the re-posting of image microblogs by users. We argue that these aggregated interactions can serve as auxiliary information to infer image interpretations. To this end, we propose a probabilistic learning framework capable of transferring such subjective information to the image-level labels based on a known aggregated distribution. We use our framework to rank images by subjective attributes from the domain knowledge of social media marketing and personality psychology. Extensive studies and visualizations show that using auxiliary information is a viable line of research for the multimedia community to perform subjective attributes prediction.
... Food computing collects data from multiple sources and involves tasks such as perception, recognition, retrieval, recommendation, prediction and monitoring of food intake. One of the key outcomes of food computing is understanding the relationship between dietary choices and health state [10,12]. A healthy diet promotes overall well-being and lowers the risk of chronic diseases. ...
... A healthy diet promotes overall well-being and lowers the risk of chronic diseases. To aid in building a healthy diet, algorithms can potentially compute health scores for food items based on the users health status [10,12], the item nutritional features, along with context and other environmental factors [11]. However, healthy food suffers from the adoption problem, since healthy food can be in conflict with taste preferences [5]. ...
... MADiMa '19, October 21, 2019, Nice, France real world implementation for enjoyment or health. Effectively extending the food recommendation to incorporate the individual health state criteria and culinary flavour and user preferences will be the next evolution of more personalized food recommendation [4,6,7,10,12]. Qualitative analysis based on user feedback will also be essential to improving quality of recommendations. ...
We propose a mechanism to use the features of flavour to enhance the quality of food recommendations. An empirical method to determine the flavour of food is incorporated into a recommendation engine based on major gustatory nerves. Such a system has advantages of suggesting food items that the user is more likely to enjoy based upon matching with their flavour profile through use of the taste biological domain knowledge. This preliminary intends to spark more robust mechanisms by which flavour of food is taken into consideration as a major feature set into food recommendation systems. Our long term vision is to integrate this with health factors to recommend healthy and tasty food to users to enhance quality of life.
... This paper proposes mechanisms for personal health monitoring by using location data, health parameters, and public sensor data streams. This is only one component of the larger exposome that an individual would have computed with modern systems for health state estimation [19] [17]. This could be used to understand the causal reasons behind changes in respiratory or cardiovascular health by the user, a medical professional, or a computational system such as personal health navigators [18]. ...
The health effects of air pollution have been subject to intense study in recent decades. Exposure to pollutants such as airborne particulate matter and ozone has been associated with increases in morbidity and mortality, especially with regards to respiratory and cardiovascular diseases. Unfortunately, individuals do not have readily accessible methods by which to track their exposure to pollution. This paper proposes how pollution parameters like CO, NO2, O3, PM2.5, PM10 and SO2 can be monitored for respiratory and cardiovascular personalized health during outdoor exercise events. Using location tracked activities, we synchronize them to public data sets of pollution sensors. For improved accuracy in estimation, we use heart rate data to understand breathing volume mapped with the local air quality sensors via constant GPS tracking.
