Ramesh Jain

University of California, Irvine | UCI · Donald Bren School of Information and Computer Sciences

Background Modern environmental health research extensively focuses on outdoor air pollutants and their effects on public health. However, research on monitoring and enhancing individual indoor air quality is lacking. The field of exposomics encompasses the totality of human environmental exposures and its effects on health. A subset of this exposo...

It is well understood that an individual's health trajectory is influenced by choices made in each moment, such as from lifestyle or medical decisions. With the advent of modern sensing technologies, individuals have more data and information about themselves than any other time in history. How can we use this data to make the best decisions to kee...

BACKGROUND Modern environmental health research extensively focuses on outdoor air pollutants and their effects on public health. However, research on monitoring and enhancing individual indoor air quality is lacking. The field of exposomics encompasses the totality of human environmental exposures and its effects on health. A subset of this exposo...

A personal food model (PFM) is essential for high-quality food recommendation systems to enhance health and enjoyment. We can build such models using food logging platforms that capture the users’ food events. As proposed in the Westermann and Jain event model, capturing six facets of multi-modal data provides a holistic view of any event. Five of...

The use of face masks is recommended worldwide to reduce the spread of COVID-19. A plethora of facial coverings and respirators, both commercial and homemade, pervade the market, but the true filtration capabilities of many homemade measures against the virus are unclear and continue to be unexplored. In this work, we compare the following masks in...

Food is central to life. Food provides us with energy and foundational building blocks for our body and is also a major source of joy and new experiences. A significant part of the overall economy is related to food. Food science, distribution, processing, and consumption have been addressed by different communities using silos of computational app...

The use of face masks is recommended worldwide to reduce the spread of COVID-19. A plethora of facial coverings and respirators, both commercial and homemade, pervade the market, but the true filtration capabilities of many homemade measures against the virus are unclear and continue to be unexplored. In this work, we compare the efficacy of the fo...

Modern health research is increasingly separated into various omics fields. Omics provides a quantified approach to the total collective information about the various realms of biology. One aspect of omics called the exposome relates to the collective exposures an individual is placed in throughout their life. A subset of this exposome is the atmos...

Sleep is critical to leading a healthy lifestyle. Each day, most people go to sleep without any idea about how their night's rest is going to be. For an activity that humans spend around a third of their life doing, there is a surprising amount of mystery around it. Despite current research, creating personalized sleep models in real-world settings...

Knowing the state of our health at every moment in time is critical for advances in health science. Using data obtained outside an episodic clinical setting is the first step towards building a continuous health estimation system. In this paper, we explore a system that allows users to combine events and data streams from different sources to retri...

With the exponential growth in the usage of social media to share live updates about life, taking pictures has become an unavoidable phenomenon. Individuals unknowingly create a unique knowledge base with these images. The food images, in particular, are of interest as they contain a plethora of information. From the image metadata and using comput...

A growing proportion of the global population is becoming overweight or obese, leading to various diseases (e.g., diabetes, ischemic heart disease and even cancer) due to unhealthy eating patterns, such as increased intake of food with high energy and high fat. Food recommendation is of paramount importance to alleviate this problem. Unfortunately,...

We propose a mechanism to use the features of flavour to enhance the quality of food recommendations. An empirical method to determine the flavour of food is incorporated into a recommendation engine based on major gustatory nerves. Such a system has advantages of suggesting food items that the user is more likely to enjoy based upon matching with...

Food is essential for human life and it is fundamental to the human experience. Food-related study may support multifarious applications and services, such as guiding human behavior, improving human health, and understanding the culinary culture. With the rapid development of social networks, mobile networks, and Internet of Things (IoT), people co...

The health effects of air pollution have been subject to intense study in recent decades. Exposure to pollutants such as airborne particulate matter and ozone has been associated with increases in morbidity and mortality, especially with regards to respiratory and cardiovascular diseases. Unfortunately, individuals do not have readily accessible me...

Events are fundamental for understanding how people experience their lives. It is challenging, however, to automatically record all events in daily life. An understanding of multimedia signals allows recognizing events of daily living and getting their attributes as automatically as possible. In this paper, we consider the problem of recognizing a...

A growing proportion of the global population is becoming overweight or obese, leading to various diseases (e.g., diabetes, ischemic heart disease and even cancer) due to unhealthy eating patterns, such as increased intake of food with high energy and high fat. Food recommendation is of paramount importance to alleviate this problem. Unfortunately,...

In this work, we explore the effects of flavour on the quality of recommendations. We attempt to build an empirical method to determine the flavour of food and incorporate them into a recommendation engine. This is particularly advantageous when combined with a system that proactively suggests healthier alternatives to common food items. Such a sys...

Health and well-being are shaped by how lifestyle and the environment interact with biological machines. A navigational paradigm can help users reach a specific health goal by using constantly captured measurements to estimate how their health is continuously changing and provide actionable guidance.

Lifestyle and environment interacting with our biological machine are primarily responsible for shaping our health and wellbeing. Continuous, multi-modal, and quantitative approaches to understanding and controlling these factors will allow each person to better reach their desired quality of life. A navigational paradigm can help users towards a s...

A food journal is essential for improving health and well-being. However, journaling every meal is extremely difficult because it depends on user initiative and intervention. Current approaches to food journaling are both potentially inaccurate and tedious, causing people to abandon their journals very soon after they start. In this paper, we propo...

Research in multimedia and health is driven by the current technological advancements in sensors and personalized healthcare. There is an increasing amount of work that shows how core multimedia research is becoming an important enabler for solutions with applications and relevance for the societal questions of health. This workshop brings together...

Deep learning has been successfully explored in addressing different multimedia topics recent years, ranging from object detection, semantic classification, entity annotation, to multimedia captioning, multimedia question answering and storytelling. Open source libraries and platforms such as Tensorflow, Caffe, MXnet significantly help promote the...

Road conditions affect both machine and human powered modes of transportation. In the case of human powered transportation, poor road conditions increase the work for the individual to travel. Previous estimates for these parameters have used computationally expensive analysis of satellite images. In this work, we use a computationally inexpensive...

Advances in user interfaces, pattern recognition, and ubiquitous computing continue to pave the way for better navigation towards our health goals. Quantitative methods which can guide us towards our personal health goals will help us optimize our daily life actions, and environmental exposures. Ubiquitous computing is essential for monitoring thes...

Food is essential for human life and it is fundamental to the human experience. Food-related study may support multifarious applications and services, such as guiding the human behavior, improving the human health and understanding the culinary culture. With the fast development of social networks, mobile networks, and Internet of Things (IoT), peo...

What if an app could guide you to better health, similar to how GPS navigation directs you to your desired destination? What if the app could use real-time information to redirect you around a disease, just as you're rerouted to avoid traffic? What if the app could provide step-by-step directions to get you to your optimal health state, whether you...

Each individual is a unique entity. We consider this uniqueness to be an important precondition for turning self tracking tools into perpetual health management systems. The tracking data itself cannot describe this uniqueness. However, actionable information based on a personal model which re ects these unique characteristics would have a strong i...

The goal of the MMHealth 2017 Workshop on Multimedia for Personal Health and Health Care was to explore the relevance and contribution of multimedia in healthcare and personal health. After a successful debut in Amsterdam in 2016, the second workshop at ACM Multimedia 2017 in Mountain View, California, again attracted more than 30 participants, ind...

The objective of this special issue is to revisit how sensor technology is transforming the way context and human behavior is understood, enabling personal and enriched media experiences.

In the recent years, digitalization of disaster management has gained much importance in aspects of response, recovery and mitigation. Often, issues identified in past disaster response and relief efforts include lack of communication, delayed ordering of actions (e.g., evacuations), and low levels of preparedness by authorities during disasters. A...

In this panel, we attempt to review and discuss the recent emerging theoretical and technological advances and trends of cross-media. Integrating data-driven machine learning with human knowledge can effectively lead to explainable, robust, and general models. Thus, the effective employment of the interaction between cross-media data during inferen...

Multimodal data streams are essential for analyzing personal life, environmental conditions, and social situations. Since these data streams have different granularities and semantics, the semantic gap becomes even more formidable. To make sense of all the multimodal correlated streams we must first synchronize them in the context of the applicatio...

Ever since the emergence of digitization, we've used the term multimedia to represent a combination of different kinds of media types, such as images, audio, and videos. As new sensing technologies emerge and are now becoming omnipresent in daily lives, the definition, role and significance of multimedia is changing. Multimedia now represents the m...

Dietary choices are the primary determinants of prominent diseases such as diabetes, heart disease, and obesity. Human health care providers, such as dietitians, cannot be at the side of every user at all times to manually guide them towards optimal choices. Automated adaptive guidance fused with expert knowledge can use multimedia data to technolo...

A root cause of chronic disease is a lack of timely informed decision power in everyday lifestyle choices, such as in diets. Users are unable to clearly delineate and demand healthy food in a quantitative manner. To scale the benefit of health nutrition coaching in broad real-world scenarios, we need a technological solution that is constantly able...

Photos are becoming spontaneous, objective, and universal sources of information. This paper explores evolving situation recognition using photo streams coming from disparate sources combined with the advances of deep learning. Using visual concepts in photos together with space and time information, we formulate the situation detection into a semi...

Managing health lays the core foundation to enabling quality life experiences. Modern multimedia research has enhanced the quality of experiences in fields such as entertainment, social media, and advertising; yet lags in the health domain. We are developing an approach to leverage multimedia systems for human health. Health is primarily a product...

Future health ecosystems demand the integration of emerging data technology with an increased focus on preventive medicine. Cybernetics extracts the full potential of data to serve the spectrum of health care, from acute to chronic problems. Building actionable cybernetic navigation tools can greatly empower optimal health decisions, especially by...

Specifying the search space is an important step in designing multimedia annotation systems. With the large amount of data available from sensors and web services, context-aware approaches for pruning search spaces are becoming increasingly common. In these approaches, the search space is limited by the contextual information obtained from a fixed...

All cities can be seen as a collection of commons. Commons are spaces, services and resources shared by the citizenry. Any change in the state of any one of the commons could lead to a potential action or a cascade of actions. In this paper we illustrate the use of a platform with three components: 1) Eventshop, an event processor, 2) Krumbs a micr...

Photos are becoming spontaneous, objective, and universal sources of information. This paper develops evolving situation recognition using photo streams coming from disparate sources combined with the advances of deep learning. Using visual concepts in photos together with space and time information, we formulate the situation detection into a semi...

Among various dimension reduction techniques, Principal Component Analysis (PCA) is specialized in treating vector data, whereas Laplacian embedding is often employed for embedding graph data. Moreover, graph regularized PCA, a combination of both techniques, has also been developed to assist the learning of a low dimensional representation of vect...

In the past decade multimedia systems have started including diverse modes of data to understand complex situations and build more sophisticated models. Some increasingly common modes in multimedia are intertwined data streams from sensor modalities such as wearable/mobile, environmental, and biosensors. These data streams offer new information sou...

With the increasing capture of photos and their proliferation on social media, there is a pressing need for a more intuitive and versatile image search and exploration system. Image search systems have long been confined to the binds of the 2D legacy screens and the keyword text-box. With the recent advances in Virtual Reality (VR) technology, a mo...

With an increasing amount of diverse heterogeneous data and information, the methodology of multimedia analysis has become increasingly relevant in solving challenging societal problems such as managing emergency situations during disasters. Using cybernetic principles combined with multimedia technology, researchers can develop effective framework...

Photos are a predominant source of information on a global scale. Cluster analysis of photos can be applied to situation recognition and understanding cultural dynamics. Graph-based learning provides a current approach for modeling data in clustering problems. However, the performance of this framework depends heavily on initial graph construction...

Heterogeneous data fusion from disparate geospatial sensors has drawn increasing attention in multimedia. Unfortunately, environmental sensors are usually sparsely and preferentially located, which restricts situation recognition of geographical regions and results in uncertainty in derived inferences. Spatial interpolation is an effective way to s...

Scanning near-field PL spectroscopy was applied to study spatial variations of the emission spectra of AlGaN epilayers with AlN molar fractions between 0.3 and 0.7. Experiments were performed at 300 K with 100 nm spatial resolution. In general, photoluminescence spectra were found to be highly uniform with the peak energy deviation of 2 to 6 meV fo...

Social image tag refinement, which aims to improve tag quality by automatically completing the missing tags and rectifying the noise-corrupted ones, is an essential component for social image search. Conventional approaches mainly focus on exploring the visual and tag information, without considering the user information, which often reveals import...

Photos serve dual role. Photos are important for capturing, saving, sharing, and reminiscing memories of events and people. Modern photos, however, are becoming more spontaneous, objective, compelling, and universal reports of a moment in an event also. In this paper our focus is on millions of photos being captured as informative reports and using...

Situation recognition is the problem of deriving actionable insights from heterogeneous, real-time, big multimedia data to benefit human lives and resources in different applications. This tutorial will discuss the recent developments towards converting multitudes of data streams including weather patterns, stock prices, social media, traffic infor...

In this chapter we discuss the notion of “situations” as broadly understood across different research fields and then provide an operational definition for it.

The proposed work lies at the intersection of multiple active research areas. Here we discuss the related work in context of the problems studied. We first survey the related areas which tackle situation-related problems. Next, we paint a timeline of how the focus on recognizing different concepts (e.g., objects, events, situations) from multimodal...

Situation modeling is the process of conceptually defining what constitutes an actionable situation in the application designer’s domain. It allows the designer to externalize what she means by a specific situation of interest (e.g., an “epidemic”). Building this model in terms of conceptual building blocks rather than directly implementing them in...

The translation of the S2S (situation-to-source) models onto applications requires computational support. As discussed in Chap. 1, this requires development of newer tools and techniques which can unify data, maintain spatial semantics, and support real-time analysis. Here we discuss the data representation proposed to combine different data stream...