Conference PaperPDF Available

Let Me Help You Learn from My Meal: User-Generated Meal Photos as a Benchmark for Nutritional Estimation

Authors:

Abstract

Healthy eating often depends on individuals' nutritional knowledge and their ability to quickly gauge nutritional composition of meals. However, both of these tasks present considerable challenge, and computing tools meant to assist in these tasks continue to be cumbersome and labor-intensive. In a pilot study, we investigated the potential of promoting nutritional literacy with social computing platforms by helping individuals compare their own meals with meals captured by others. We compared this holistic comparison approach with a more traditional method that requires mental decomposition of a meal into ingredients and estimating their portion sizes in a controlled experiment with crowd workers on Amazon Mechanical Turk. Based on the results of this study, we have identified new approaches for incorporating socially generated data, crowdsourced evaluations, and gameful components into smartphone apps to increase engagement and fun.
Let Me Help You Learn from My Meal: User-Generated Meal Photos
as a Benchmark for Nutritional Estimation
Healthy eating often depends on individuals’ nutritional knowledge and their ability to quickly gauge
nutritional composition of meals. However, both of these tasks present considerable challenge, and
computing tools meant to assist in these tasks continue to be cumbersome and labor-intensive. In a pilot
study, we investigated the potential of promoting nutritional literacy with social computing platforms by
helping individuals compare their own meals with meals captured by others. We compared this holistic
comparison approach with a more traditional method that requires mental decomposition of a meal into
ingredients and estimating their portion sizes in a controlled experiment with crowd workers on Amazon
Mechanical Turk. Based on the results of this study, we have identified new approaches for
incorporating socially generated data, crowdsourced evaluations, and gameful components into
smartphone apps to increase engagement and fun.
Learning-by-Example Paradigm
As early as the mid 1950s, cognitive scientists have adopted the learning-by-example’ paradigm and
have shown consistent interest in example-based learning where learning occurs based on a single or
few training examples. Learners solve problems through analogies, meaning they refer to known
examples and try to relate them to the problem of interest that needs to be solved (Anderson et al,
1997).
In this study we tested the effectiveness of a novel ‘learning-by-example’ paradigm for assessing
nutritional composition of meals in user-generated meal photographs, compared to a standard diabetes
education strategy for nutritional evaluation. In the ‘learning-by-example’ paradigm, individuals compare
their own meals to other users’ meal photographs with known nutritional composition. They then use
these meals submitted by others as benchmarks for estimating nutrition in their own meal. The
traditional strategy requires individuals to assess meals by decomposing and estimating portion sizes
and macronutrient content in a stepwise procedure. In a between-subjects pilot experiment with Amazon
Mechanical Turk, we found both strategies to be effective. However, the novel ‘learning-by-example’
strategy had significantly greater influence on learning and knowledge transfer, while the traditional
strategy showed greater accuracy in nutritional estimation while the strategy was in use in real-time.
Learning from Socially Generated, Crowdsourced Data
Based on the pilot’s discoveries, we started investigating applicable computational methods for future
research opportunities so that more users in their daily lives can enrich their nutritional learning from
socially-generated data. We are searching for solutions to merge the ‘learning-by-example’ approach
with data science to address the issue of learning at scale at a low cost while keeping high user
engagement. In order to accomplish this, we have taken the first steps of designing a crowdsourced,
data-driven user-interface leveraging theories from learning sciences and gamification. This approach
has a number of benefits and opportunities to create unique and optimal learning experiences. First,
users can nutritionally evaluate their real-life meal photos on demand by comparing them to other users’
meals that have been evaluated socially. Second, social computing platforms can encourage the culture
of ‘learning together’ through helping users take ownership of their evaluated meal photos by sharing
them with the community, receiving feedback from peers and points on those assessments, and
developing a reputation and status for nutritional assessment.
Design of an mHealth App, MealMe
Our challenge remains: designing and developing an interactive informatics solution that delivers health
content in social settings in an effective and motivationally intriguing way. We have prototyped a design
for an mHealth app, MealMe, that aims to help the average user estimate nutrition in their own meals
compared to other users’ meals in a lightweight, gameful approach. We are leveraging player’s sense of
presence via the role of avatars (Lim & Reeves, 2009) by incorporating avatars that will consume the
player’s real-life meals. We plan to continue iterating the design and development of MealMe as we
consider methods of incorporating socially generated data and defining the nutritional ground truth for
real-life meals that will be nutritionally assessed by non-experts, average users, in the wild.
References:
Anderson, J. Finchman, J. and Douglass, S. (1997). The role of examples and rules in the acquisition of a cognitive skill. Journal
of Experimental Psychology: Learning, Memory, and Cognition, 23(4), 932.
Lim, S., and Reeves, B. (2009). Being in the game: Effects of avatar choice and point of view on psychophysiological responses
during play. Media Psychology, 12(4), 348-370.
... Simultaneously, work in social computing has pointed to the value of crowdsourcing as a mechanism to support nutritional learning. Several studies have shown individuals are often willing and eager to share their nutrition knowledge to support the education of the community [9,29,33]. Furthermore, studies have demonstrated that users are particularly receptive to nutrition suggestions from others [9,19,25,42] and excited to learn from community knowledge [45] and translate this knowledge into behavioral decision making [35]. ...
Article
The innovative features of multi-player computer games offer compelling opportunities for self-representation during interactions, and the ways in which these avatars are chosen and manipulated may change interactive experiences. This study investigated the effects of avatar choice (choice vs. no choice) and visual point of view (POV; first-person vs. third-person) on the physiological arousal and subjective evaluations of game experiences. A 2 (Avatar Choice, No Avatar Choice) × 2 (first-person POV, third-person POV) × 2 (female players, male players) mixed-design experiment was conducted (N = 22). The results demonstrated that being able to pick the character that will represent the player in the game leads to greater arousal, especially for males. Visual POVs alone did not affect the game player's arousal, but moderated the effect of avatar choice on the game player's heart rates. Avatar choice produced a more pronounced effect in the third-person POV (where the “camera” was located behind the avatar) in which avatar choice was visually more reinforced than in the first-person POV (where the “camera” was the eyes of the avatar). The results also revealed that the gender of the game player was a significant factor in game play experience. The results suggest theoretical implications of video game self-representation and effects on game player's psychophysiological responses.
The role of examples and rules in the acquisition of a cognitive skill
  • J Anderson
  • J Finchman
  • S Douglass
Anderson, J. Finchman, J. and Douglass, S. (1997). The role of examples and rules in the acquisition of a cognitive skill. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23(4), 932.