PreprintPDF Available

Abstract and Figures

Activity in Human-Food Interaction (HFI) research is sky-rocketing across a broad range of disciplinary interests and concerns. The dynamic and heterogeneous nature of this emerging field presents a challenge to scholars wishing to critically engage with prior work, identify gaps and ensure impact. It also challenges the formation of community. We present a Systematic Mapping Study of HFI research and an online data visualisation tool developed to respond to these issues. The tool allows researchers to engage in new ways with the HFI literature, propose modifications and additions to the review, and thereby actively engage in community-making. Our contribution is threefold: (1) we characterize the state of HFI, reporting trends, challenges and opportunities; (2) we provide a taxonomy and tool for diffractive reading of the literature; and (3) we offer our approach for adaptation by research fields facing similar challenges, positing value of the tool and approach beyond HFI. CCS CONCEPTS • Human-centered computing → HCI theory, concepts and models. * Altarriba Bertran and Wilde are co-first authors.
Content may be subject to copyright.
Making Sense of Human-Food Interaction
Ferran Altarriba Bertran
Social and Emotional Technology Lab
UC Santa Cruz
Santa Cruz, CA
ferranaltarriba@gmail.com
Samvid Jhaveri
Social and Emotional Technology Lab
UC Santa Cruz
Santa Cruz, CA
samvid@ucsc.edu
Rosa Lutz
Social and Emotional Technology Lab
UC Santa Cruz
Santa Cruz, CA
rolutz@ucsc.edu
Katherine Isbister
Social and Emotional Technology Lab
UC Santa Cruz
Santa Cruz, CA
katherine.isbister@ucsc.edu
Danielle Wilde*
SDU Design
University of Southern Denmark
Kolding, Denmark
wilde@sdu.dk
ABSTRACT
Activity in Human-Food Interaction (HFI) research is sky-
rocketing across a broad range of disciplinary interests and
concerns. The dynamic and heterogeneous nature of this
emerging eld presents a challenge to scholars wishing to
critically engage with prior work, identify gaps and ensure
impact. It also challenges the formation of community. We
present a Systematic Mapping Study of HFI research and an
online data visualisation tool developed to respond to these
issues. The tool allows researchers to engage in new ways
with the HFI literature, propose modications and additions
to the review, and thereby actively engage in community-
making. Our contribution is threefold: (1) we characterize the
state of HFI, reporting trends, challenges and opportunities;
(2) we provide a taxonomy and tool for diractive reading of
the literature; and (3) we oer our approach for adaptation
by research elds facing similar challenges, positing value
of the tool and approach beyond HFI.
CCS CONCEPTS
Human-centered computing
HCI theory, concepts
and models.
Altarriba Bertran and Wilde are co-rst authors.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies
are not made or distributed for prot or commercial advantage and that
copies bear this notice and the full citation on the rst page. Copyrights
for components of this work owned by others than the author(s) must
be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specic
permission and/or a fee. Request permissions from permissions@acm.org.
CHI 2019, May 4–9, 2019, Glasgow, Scotland UK
©
2019 Copyright held by the owner/author(s). Publication rights licensed
to ACM.
ACM ISBN 978-1-4503-5970-2/19/05.. .$15.00
https://doi.org/10.1145/3290605.3300908
KEYWORDS
Human-Food Interaction, Literature Review, Scientic Map-
ping Tools, Data Visualization
ACM Reference Format:
Ferran Altarriba Bertran, Samvid Jhaveri, Rosa Lutz, Katherine
Isbister, and Danielle Wilde*. 2019. Making Sense of Human-Food
Interaction. In CHI Conference on Human Factors in Computing
Systems Proceedings (CHI 2019), May 4–9, 2019, Glasgow, Scotland
UK. ACM, New York, NY, USA, 13 pages. https://doi.org/10.1145/
3290605.3300908
1 INTRODUCTION
The dynamic and heterogeneous nature of HFI as a eld
presents a challenge to scholars wishing to critically engage
with prior work, position their research in context, identify
gaps and ensure impact. We therefore decided to undertake a
comprehensive review in the form of a Systematic Mapping
Study (SMS) [10].
Our SMS began with data collection. The aim was to create
a comprehensive dataset of scientic publications, to better
understand the scope of HFI. Reviews, both within and be-
yond HCI, often use computation to collect publications au-
tomatically from online repositories [
16
]. Relevant examples
include a visual analysis approach to update systematic re-
views [
28
] and a framework for conducting reviews through
web scraping and data clustering [
49
]. Automated scraping
of data from digital repositories is a useful technique when
the scope of a research eld can be determined, and data
points categorized through a solid rationale. Unfortunately,
this was not possible in HFI.
HFI is an emerging research eld that is remarkably dy-
namic and heterogeneous. It cuts across numerous research
disciplines and is approached from varied perspectives, re-
ecting the diversity of ways that people interact with food.
Because of this diversity, we struggled to determine where
an algorithm should search for HFI publications and what
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 1
exactly it should look for. We knew the ACM database in-
cludes HFI-related publications. We also knew we would nd
important publications elsewhere. Despite concerted eort,
without a comprehensive dataset, we were unable to develop
a solid rationale to categorize data points algorithmically
and we failed to formulate a thematic structure to organize
the not-yet-gathered, burgeoning dataset. We were also un-
able to nd an existing framework of HFI that might help us.
We therefore decided to build a comprehensive dataset and
develop a structure for categorisation using a fully manual
method, following Bar-Ilan and Aharony [7].
To build the dataset, we began with a keyword, title and
abstract search of ’food’ in the ACM database. We manually
ltered the rst 700 results, to guard against repetition and
exclude papers we determined did not belong. We then added
texts from other sources, based on author knowledge and
appearance in references. The rst author has experience as
a designer and researcher in gastronomic restaurants. The
last author’s research cuts across Design, HCI and Science,
Technology and Society Studies (STS). Together, they share
a long-standing interest in food design and experimental
food practices. This diverse experience enriched our under-
standing of where to look for sources and enabled us to
compose a dataset that includes papers from a broad spec-
trum of venues and repositories. This approach required
careful negotiation to determine t and pointed to a need to
validate determinations with the research community. The
resulting dataset—before external evaluation—consisted of
260 publications, most published since 2009 (Figure 1).
In this paper we report the emergent process and outcomes
of our SMS. Our contribution is threefold: (1) We characterize
the current state of HFI to empower interested researchers
to better position their work and nd new opportunities
for impact. (2) We provide an online data visualisation tool
1
to allow the research community to dynamically update
1The tool can be accessed at: http://www.ferranaltarriba.com/h
Figure 1: HFI publications per year, using our initial dataset.
our review, and thus engage actively with the dynamic and
heterogenous nature of HFI. (3) We propose our process and
tool for adaptation by other, equally uid, emerging elds
of research that struggle with similar issues. We thus posit
value for our process and tool beyond HFI.
2 DATA ANALYSIS
We began our analysis using a combination of quantitative
and qualitative methods, inspired by Halskov and Hansen
[
31
]. We rst examined two quantitative variables: publica-
tion year and venue. The results conrmed our impression
that the research space of HFI is not only emergent, but can
be characterised by radical growth (see Figure 1). Our analy-
sis also indicated remarkable heterogeneity. The rst dataset
includes papers across 76 dierent venues. A variety of dis-
ciplines, foci of interest and methodological approaches are
represented, including: cross-modal psychology (e.g. [
59
]),
engineering (e.g. [
37
]), computer science (e.g. [
47
]), HCI (e.g.
[
43
]), speculative design (e.g. [
25
]), and more. Further, we
found that a signicant number of papers are disseminated
through workshops and Special Interest Groups (SIGs).
We also identied three growing communities within HFI.
The Food CHI community originated in 2017 as a SIG [36] and
was maintained in 2018 as a conference workshop [
26
]. It ex-
tends previous attempts at community-making around food
and interaction design (e.g. [
17
,
18
]). Examples of research
in this community include: Choi et al.’s edited collection of
CHI research into eating, cooking and growing food [
13
],
Dolejšová and Kaiying’s research that encourages citizens
to co-design DIY food-related technologies [
24
], and Khot et
al.’s 3D chocolate print system that oers users personalised
’activity treats’ after exercising [39].
The second community, Multisensory Human-Food Interac-
tion (MHFI), has been gathering in workshops since 2016 (e.g.
[
50
,
63
]). Well-cited research from this community includes:
Obrist’s account of multisensory interfaces [
51
] and Spence’s
Gastrophysics investigations of the impact of multisensory
phenomena on people’s eating behaviour [58].
The third community seems less cohesive. Since 2012, com-
puter scientists have discussed AI approaches to HFI at a vari-
ety of workshops, including Cooking with Computers [
20
,
21
],
Multimedia Cooking and Eating Activities [
1
,
2
,
36
], and Multi-
media Assisted Dietary Management [
3
]. Examples of papers
from this community include: Hashimoto et al.’s multimodal
method for recognizing ingredients in food preparation [
34
]
and Mori et al.’s machine learning approach to recipe text
processing [
44
]. Although few researchers participate across
all of these workshops, our analysis suggests a strong con-
nection across the research concerns. We thus consider them
a distinct community.
According to our initial dataset, researchers connected to
these three communities are among the most active in HFI;
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 2
the HFI community tends to advance in sub-communities
gathered around workshops and SIGs, with little crossover
between the three communities.
Following a quantitative analysis based on publication
year and venue, we conducted a qualitative analysis of the
data. Our aim was to identify anities and dierences, spot
trends and unexplored opportunities. To ground our analysis,
we intended to use a theory informed approach to categorise
the data [
31
]. Yet, HFI is such a new eld that we could not
nd a community-accepted theoretical frame to follow. We
needed a clear, transversal set of criteria to map the publica-
tions and thus characterize the eld beyond the surface. We
therefore developed a taxonomy from the data itself.
3 A CONCEPTUAL MODEL OF HFI RESEARCH
Taxonomies simplify complex realities. While their use im-
plies a compromise on nuance, they aord a structure to
make sense of data that would otherwise be dicult to grasp.
The rationale behind our taxonomy was shaped by a need to
(1) embrace the diversity of perspectives included in the HFI
publication dataset, and (2) reect contemporary issues in
HCI, such as the multiplicity of roles of technology in human
life [
33
] and the balance between automation and individual
empowerment [
38
]. This was an emergent process: as we
engaged the data in more detail, the coding evolved until
we arrived at the taxonomy. The result includes three lenses
through which HFI publications can be analysed:
Focus
aords positioning of HFI papers on a continuum
between functionality and experience, where the latter is di-
vided across individual experience and social bonding. This
lens responds to the variety of purposes we found in the
dataset. For example, eating monitoring systems seem to
be viewed as functionality-oriented artefacts with a clear
instrumental role (e.g. [
67
]). In contrast, Multisensory HFI
contributions often focus on augmenting individual expe-
rience of food, e.g. through sound stimuli [
62
]. Dierently,
many speculative works propose food as a platform for so-
cial bonding, supporting and augmenting social interaction
around food practices, e.g. digital food sharing [56].
Agency
refers to the interplay between humans and tech-
nology when dealing with food. To represent the diversity of
approaches to agency we found in the dataset, publications
are attributed a position on a continuum between person and
technology, depending on how the researchers themselves
determine that their work attributes agency. On one end of
the spectrum, we nd artefacts that perform food-related
tasks with a high degree of autonomy from humans. For
example, a pair of eyeglasses that track chewing to monitor
eating activity without user input [
68
], or a recipe-generation
algorithm [
47
]. On the other end, we nd contributions that
empower humans to conduct food-related practices them-
selves; for example, an exploration of food democracy in
local food networks [
54
], or a study of user experience in
user-managed food journaling systems [19]).
Domain
responds to the diverse nature of human-food
relationships, present in many areas of life, by enabling cate-
gorisation of research in relation to the human-food web. A
rst pass at the data identied seven domains: source, store,
produce, eat, identify, waste and track. A closer read led us
to conate 1) store and waste, and 2) identify and track, as
we noted they were often handled interchangeably. We also
added ’speculate’ to represent speculative research that did
not clearly fall into other domains. The nal lens proposes 6
categories to classify HFI contributions:
Source: foraging or buying food (e.g. [35, 41]).
Store: practices of both storing and disposing of food
(e.g. [27, 52]).
Produce: growing foods, as well as manipulating them
to create more complex combinations, such as dishes
or meals (e.g. [64, 69]).
Track: identication and measurement of food and
food practices (e.g. [42, 46]).
Eat: food consumption (e.g. [57, 60]).
Speculate: contributions that explore alternative food
futures, or conduct meta-reections on HFI as a re-
search eld (e.g. [12, 24]).
Analysing our Dataset Through the Taxonomy
Using our three-lens taxonomy, we manually sorted the 260-
publication dataset. For each publication, we determined
a position in the focus and agency continuums and a sin-
gle domain, based on our interpretation of the text. Each
publication was coded by one researcher, then validated or
contested by another. In case of disagreement, we negotiated
which code to attribute. To ensure the dataset was robust,
we shared our coding publicly and oered rst authors the
chance to contest our analysis. We received and actioned 8
requests for changes, including allowing multiple domains
per publication (see Community Validation section for more).
The taxonomy helped us bind together contributions with
radically dierent perspectives and compare papers that we
previously struggled with. A rst read of the data through
the conceptual model provided the following insights:
Functionality vs. Individual Experience vs. Social Bonding. The
majority of HFI publications focus on functionality, with less
emphasis on experiential aspects of food practice. Within
experience-related contributions, there is more emphasis
on individual experiencing of food practice than on social
bonding (Figure 2, left).
Technology vs. Person. Contributions that give greater au-
tonomy to the technology rather than the person using the
technology are more numerous than those that empower
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 3
Figure 2: First ndings across three lenses (l-r): Focus, Agency and Domain. In Focus and Agency, the x-axes represent number
of papers and the y-axes include intermediate steps between categories to aord more nuanced interpretation.
humans to impact their personal food practices (Figure 2,
centre). Speculative HFI works counter this trend.
Unequal distribution of concerns. Papers that focus on food
production (32,8%), eating (26,3%) and tracking (21.6%), dom-
inate. Sourcing (12.4%) is also relatively well explored, com-
pared to storing (3.5%) and speculating (3.5%), which seem of
little interest to the HFI community to date (Figure 2, right).
At this point, our dataset contained 260 data points, cate-
gorized through three lenses, including meta-data such as
author keywords, publication year, venue and type. This
number of variables was challenging to manage without au-
tomation. To facilitate more in-depth readings of the dataset,
we decided to develop an interactive data visualisation tool.
The aim was to leverage the potential of computation to visu-
ally map the data according to our conceptual model, taking
the meta-data variables into account. In the next section, we
describe the tool and examine how it extends previous works
on data visualisation for literature reviews.
4 DATA VISUALISATION TOOL
The use of data visualisation to characterize research elds
is not new. Data visualisations enable viewers to make sense
of datasets that would otherwise be dicult to parse. When
interactive, they aord personalised queries, and thus more
in-depth and succinct analyses. Previous systematic mapping
studies that leverage this technique include: a visualisation
of the evolution of technical games research [
48
], a tool
to support the review of HCI and InfoVis literature [
53
],
a system to visualise research trends in conferences [
40
]
and an interactive characterisation of the state of the art in
visualizing dynamic graphs [
11
]. Inspired by these examples,
we designed an online tool to allow dynamic visualisation of
our dataset using the HFI taxonomy. The resulting tool gives
access to the dataset, and the possibility to edit or propose
additions. It aords personalised readings of the data, using
lters to shift perspectives or view everything at once. A
researcher can thus gain a general overview of the state of
HFI research, or focus on dierent perspectives as they target
their analysis.
Figure 3 provides a snapshot of the tool interface, with all
lters selected and thus all publications visible. Papers are
represented as coloured dots on a graph. The lters, available
on the right, include: domain, focus, agency, publication year,
venue, keywords and type (e.g. journal article, full conference
paper, extended abstract, book, etc.). Each publication-dot
visualises domain through colour, focus through positioning
on the horizontal axis, and agency through positioning on
the vertical axis. Dots can be selected to view the associ-
ated citation, abstract and keyword information below the
visualisation area.
Our tool—which we are calling theHFI Lit Review App
enabled us to quickly and easily verify emerging trends iden-
tied during desk research. This capability was signicant,
as our ndings were somewhat intuitive when manually-
comparing 260 data points. When visualised using the HFI
Lit Review App, we could quickly determine if trends held
across the data. We could also identify trends that were previ-
ously not visible. For example, contributions in the domain of
tracking strongly tend towards technology along the agency
continuum, and in the domain of eating towards individual
experience. While, on the surface, these trends may not be
surprising, it is useful to see it in the data. The visualisation
makes apparent opportunities to expand areas of research
towards, for example, supporting increasingly social interac-
tions through food.
Figure 3: First iteration of the HFI Lit Review App.
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 4
Because of the dynamic nature of HFI research, the HFI
Lit Review App will only be useful if it is up to date, and if
there is consensus on how content is represented. We there-
fore set up the app like a wiki, with guidelines to ensure a
robust dataset. Readers can suggest modications and addi-
tions, challenge existing interpretations of data, and contrast
their own and others’ perspectives on research. Proposed
changes are monitored by a committee, currently made up
of the authors. The committee defers to author requests, and
opens up for discussion proposals made by others. This in-
built opportunity for discussion is meant to stimulate fruitful
dialogue within the HFI community. The intention is to em-
power scholars to collectively build and maintain a shared
repository of publications across the diverse approaches to
HFI. If successful, it will allow the app to become an evolving
platform and locus for community-driven sense making.
Data visualizations for systematic mapping studies are
typically automated: they often rely on computation to col-
lect, analyse and represent quantitative data. However, there
is more than one way to understand data [
55
], and in a eld
of research as heterogeneous as HFI dierences of perspec-
tive bring added value. We therefore designed the app to
enable diractive reading. As [
8
,
9
] explains, (building on
[
32
]) diractive reading maps interference, not replication,
reection or reproduction: not where dierences appear but
rather where the eects of dierence appear. By making
dierences visible, diraction aords rapid and easy iden-
tication of research opportunities. As we discuss below,
our experience suggests that diractive reading of the data
better positions researchers to challenge their assumptions
and embrace other perspectives.
Critically, our tool relies on human interpretation—interes-
ted researchers can contribute, and thereby help make sense
of HFI. As they read the data, they can change lters to
gain diering perspectives to understand other researchers’
points of view. This combination of human interpretation
and computational visualisation is a key characteristic of the
app. It enables the construction of a more comprehensive
dataset and facilitates closer, more nuanced reading of the
data. Our tool allows researchers to engage with a dataset of
literature that has been, and continues to be, categorised and
updated by the broader HFI research community. We believe
this approach has great potential in any eld where radi-
cally diverse perspectives coexist. As we discuss in the next
section, community response, to date, has been enthusiastic.
5 COMMUNITY VALIDATION
To ensure our work reects the diversity of perspectives
within HFI, we needed to validate our taxonomy and expand
the dataset in consultation with community. We also needed
to conrm whether researchers across HFI saw value in our
tool, and if they would be willing to use and maintain it.
Methodology
Our community validation strategy involved two phases:
First, we opened up our provisional work for comment at
the 2018 ACM Designing Interactive Systems conference
(DIS’18) [
4
]. Second, we contacted the rst authors of each
publication in the dataset by e-mail (except where contact
emails were missing or invalid). Both actions were framed
as an invitation to comment on our approach and propose
modications to the taxonomy, app and dataset.
DIS’18 Poster presentation. During the DIS poster session,
we held semi-structured conversations with 15 interested
researchers, using the following tangible conversation tools
[15] to elicit constructive comments:
A working version of the HFI Lit Review app on a
laptop, that researchers could test in-situ.
Thirteen printed papers from the dataset, that partici-
pants were invited to review. The papers were chosen
for their diversity, and because we had found them
challenging to categorise. For example, we struggled
to determine the domain of an artwork that explores
the human food system [
22
], and to position along the
agency continuum a system to create programmable
taste structures [
69
]. We hoped that (re)categorization
Figure 4: Community validation at DIS’18. Top-left: poster.
Top-right: researchers engaging with an analogue version
of the tool. Bottom: analogue contributions by participants:
dots placed on the matrix and sticky note comments.
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 5
of these publications by other researchers might pro-
vide a useful challenge to our conceptual model.
A poster, including the empty data visualisation frame-
work (Figure 4, top left), that researchers could ll in
and comment on. The poster included the citation and
abstract of the accompanying publication [
5
], available
on the ACM database (at this point without a paywall).
Colored sticker dots and sticky-notes that could be
placed onto the poster, as well as pens, markers and
papers for further comments.
E-mail correspondence. We sent an e-mail to all rst authors
in the dataset, oering them a chance to give feedback and
challenge our analysis of their work. The e-mail included:
A short description of our research and motivations,
and a link to the work-in-progress publication [5].
A direct link so they could easily suggest changes to
our analysis of their publication(s) in the dataset.
Four questions asking for feedback about the tool: Are
the lters we chose working for you? Is anything missing
that you consider critical? Should something be changed?
Once it’s nished, do you think it will be useful for you
or for other researchers working in this space?
At the time, the dataset included 260 publications. A num-
ber of authors had more than one paper, the maximum being
9. We accessed 129 active e-mail addresses of 212 rst authors
included in the dataset.
Results
We received feedback from 26 respondents in total: 15 at
DIS’18 and 11 through email. 7 authors—4 via email—used
the app to re-interpret the categorization of their research.
One author reinterpreted their work manually (at DIS’18).
We used thematic analysis [
30
] to examine this feedback. We
organise our ndings here in four themes: the online tool,
the dataset, our analysis and the conceptual model.
Online tool. All 26 respondents expressed enthusiasm to-
wards the potential of the HFI Lit Review tool: ‘I am very
glad to know that someone has thought about creating a data-
base for HFI’ (R8, e-mail), and: ‘I denitely think it would
be useful’ (R7, e-mail). Some respondents said they strug-
gled to position their work and nd opportunities for future
research when navigating HFI literature: ‘I’ve been doing
a review of the HFI works for one of my thesis chapters
now, and it’s tricky, right. What is HFI anyway?’ (R3, e-mail).
A number of researchers suggested that the tool could be
useful beyond HFI, pointing out that the combination of com-
putational visualisation and human interpretation, with a
custom taxonomy such as we created, might help character-
ize other emerging elds within HCI., e.g. Animal-Computer
Interaction. Another researcher (at DIS’18) noted that such
a tool would be useful for PhD candidates working at the
intersection of several elds.
When asked about the usefulness of the tool, participants
highlighted a number of benets, including getting ‘a better
overview over the publications’ (R9, e-mail) thanks to ‘such
an impressive database’ (R7, e-mail). The tool was deemed
useful at dierent levels: it allows researchers to ‘keep track
on the latest publications’ (R9, e-mail), aords ‘a nice meta-
analysis and discussion about the HCI and food literature’
(R6, e-mail), and makes it possible to ‘nd underexplored op-
portunities’ (R6, e-mail). Researchers were also enthusiastic
about the community functionalities, noting that ‘it enables
you to check back with the researchers’ (R7, e-mail).
When asked about improvements to the app, participants
suggested minor changes: adding detailed information about
the taxonomy, and graphic design renements such as high-
lighting the title of publications in the data visualisation.
They also proposed new functionalities. Some wanted to be
able to ‘zoom into the visualisation so that the individual pub-
lication points are easier to click on’ (R9, e-mail). A number
of researchers asked us to display the number of papers as-
sociated with specic clusters of data (DIS’18, conversation).
Some suggested combining manual data analysis with auto-
mated data collection, now that we have a comprehensive
list of HFI-dissemination venues.
Dataset. Participants found our dataset comprehensive. At
DIS’18, it was common that they checked the raw dataset to
nd whether it included their papers or papers they knew
about. Only one participant noted (two) missing papers. In
the e-mail responses, 8 authors suggested a total of 9 new
papers. They have since been added. Researchers also helped
identify repetitions we had missed, now corrected.
Data analysis. There were researchers who agreed with our
analyses, and others who challenged them. Most of the sug-
gested changes were minor, except one, which required a
signicantly dierent position on the agency and focus con-
tinuums, and a dierent domain. All suggested changes were
implemented by the study participants using the online tool.
Conceptual model. The taxonomy underlying the data vi-
sualisation tool seemed to resonate with all participants’
understandings of HFI, those at DIS’18 and those who re-
sponded to the e-mail request. According to researchers at
DIS’18, the lenses respond to relevant issues. Some noted
that these same lenses could be useful beyond HFI, to discuss
the interplay between humans and technology more broadly.
Overall, participants said they found the taxonomy appro-
priate, though some noted issues that could be addressed
moving forward. Some suggested allowing the choice of mul-
tiple domains, as many publications refer to more than one
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 6
area of food-practice (e.g. producing and eating). Some sug-
gested that the eating domain was not ‘on the same level
as the others’ (R1, e-mail). They ‘could see many dierent
styles of papers getting that lter’ (R6, e-mail). R1 proposed
‘consume’ as an alternative name for this category. Some
participants suggested rethinking the Focus lens, using a
dierent encoding to allow representation of papers that
refer to ‘all sorts of dimensions simultaneously’ (R7, e-mail).
R3 suggested adding a new domain ‘or some other indicator
referring to ’methods” to ‘question what we do in HFI’ and
‘challenge the status quo of food-tech design much more
than we’re doing right now though our HFI works’. Finally,
R7 wondered whether it was possible to open ‘space for [...]
post-human approaches.
Changes to the Online Tool
Inspired by this feedback, we implemented a number of
changes to the HFI Lit Review App. On an interface level,
we xed typos and minor bugs spotted by participants and
added detailed information about the taxonomy. For the con-
ceptual model, we allowed for multiple domains, adjusting
both the database and the visualisation accordingly. We also
created a list of future changes to implement, moving for-
ward. We describe those changes, as well as how we intend
to address them, in the Future work section.
Figure 5: Screenshots of the data visualization, displaying
individual readings of the categories in the lens of agency.
6 CHARACTERISING HFI
After the community validation process, we ran a nal round
of data analysis using the latest version of the tool and the
updated dataset (270 publications). Our aims were twofold:
i) to illustrate how the tool can be used and ii) to validate if
the trends identied initially still held, following the recent
changes. Our analysis oers general insights—an overview
rather than an in-depth review.
We rst tested the robustness of trends found in our ini-
tial analysis. We conrmed the radical growth of HFI: 6% of
papers in the dataset were published before 2009; 51% in the
following six years (2010-15); 43% in the last two and a half
years, (2016-mid 2018). When looking at the Agency lens
(Figure 5), human-centric papers seemed to dominate some-
what, though it was not clear if this was actually the case.
A quantitative analysis revealed that contributions that at-
tribute agency to technology (52%) are slightly more common
Figure 6: Screenshots of the data visualization, displaying
individual readings of the categories in the lens of focus.
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 7
Figure 7: Screenshots of the data visualization, displaying individual readings of the dierent categories in the lens of domain.
than those that use technology to support people in their
agency (48%). A high concentration of technology-centric
contributions in the track domain, resulting in a dense clus-
ter of blue dots in a small area (Figure 5a, top-left) seems
to prompt this misreading. This discrepancy conrms what
many HFI researchers noted: there is value in combining the
qualitative input oered by the visualisation with quantita-
tive data. When examining the data through the Focus lens
(Figure 6), we noted that functionality-oriented papers (66%)
outweigh experience-related ones (34%). Similarly, research
on individual experience of food (22%) is more common than
that on social aspects of food practice (12%).
Despite having modied the app to allow data points with
more than one domain—enabling the analysis of contribu-
tions that are concerned with multiple areas of food practice
at once—we could still conrm that there is an unequal dis-
tribution of concerns within HFI (see Figure 6). We found
that production of food is the most researched space (37%),
followed closely by eating (30%) and tracking (23%). To a
lesser extent, contributions related to sourcing (15%) are also
present in the dataset. Those categorized as speculate (6%)
and store (5%) are in the minority.
Domain-Specific Trends
After verifying the trends from our early analysis, we con-
ducted a more in-depth dive into the data. We examined the
six human-food interaction domains to identify trends and
opportunities for research. We report our ndings here.
Source. Most papers exploring food sourcing embrace an
instrumental idea of technology (Figure 7a), focusing on an
artefact’s functionalities rather than on the experience of
interacting with it. In terms of agency, we see a balance
between contributions that leverage technology automation
(e.g. a meal recommendation algorithm [
66
]) and those that
empower users to source food themselves (e.g. a system to
support urban foraging [
23
]). Some source papers are co-
categorised in the produce domain.
Store. Similar to the source domain, publications categorised
in the store domain tend to be functionality-oriented (Figure
7b). Most data points are situated in the middle of the agency
continuum, and combine technology automation with user
empowerment. For example, a study of food waste reduction
systems that leverage color-coding and camera tracking to
encourage and support behaviour change [27].
Produce is the dominant domain in the dataset. It presents
a higher diversity of foci than other domains (Figure 7c).
However, functionality-oriented research still outweighs
experience-related research. Signicantly, we see a corre-
lation between a produce publication’s approach to agency
and its position on the focus continuum. Research concerned
with experiential and social aspects of food production tend
to embrace a more human-centric stance to HFI than a func-
tionality oriented stance. Some produce papers are co-catego-
rised in the source and/or eat domains.
Eat may be the most balanced domain in terms of focus
(Figure 7d). However, papers with a social approach are still
less common than those concerned with individual eating
experience. This outcome is explained by the fact that many
works in the eat domain belong to the Multisensory HFI
community, whose focus is to unpack the impact of multi-
sensory stimuli on the perception of taste at an individual
level. In terms of agency, technology-centric contributions
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 8
dominate (e.g. a pervasive game to modify eating habits [
37
]).
However, there are also a number of papers that embrace
a more human-centric stance to eating-related HFI (e.g. a
study on the opportunities for playful social interactions in
ne dining [
6
]). A number of papers in the eat domain are
co-categorised as source,produce and/or track.
Track. As indicated by the cluster of blue dots on Figure 7e,
track is the domain that leans more towards functionality-
oriented, technology-centric approaches to HFI. Track papers
often embrace an instrumental idea of technology, focusing
on functionality rather than experiential or social factors, e.g.
a multi-device food logging system that supports nutritional
tracking [
57
]. While there are exceptions to this rule (e.g.
a study of online social dynamics related to healthy eating
[
14
]), they remain limited. Publications in the track domain
are sometimes co-categorised as store and/or produce.
Speculate. Although speculative approaches to HFI seem
to be gaining traction, they are still in the minority. Most
research in this domain has been published since 2015. Spec-
ulate shows a dierent landscape than other domains (see
Figure 7f). Contrary to technology-centric trends, most spec-
ulative research investigates how humans might gain more
agency in their relationship with food. Many publications in
this domain focus on social phenomena surrounding food
practice. For example, [
24
] encourages citizens to co-design
DIY low-cost technology to support "smart" food practices.
Papers in the speculate domain are often co-categorised as
source,store or produce.
7 DISCUSSION
A Living Set of Tools for Community Sense-Making
The key contribution of our Systematic Mapping Study are
the HFI Lit Review App and the underlying conceptual model.
Our goal is not to achieve a detailed characterisation of HFI,
but rather provide conceptual and operational tools so that
HFI researchers can engage with the literature and come to
conclusions that reect their own concerns. Rather than a
literature review presented in linear form, we oer a set of
tools that aord diractive reading of HFI research. Dirac-
tive reading not only acknowledges dierence, but values
its eects—it supports divergent perspectives and thereby
accommodates the inherent heterogeneity of HFI. Rather
than perform sense-making, our tools support its emergence
by enabling diverse community actors to: (1) make their own
sense of what they discover, and (2) collectively contribute
to higher level reections on the eld.
Our contribution is a foundation from which to make
sense, rather than a static review that weaves a particular
view of the current state of the art. We argue that such a
move is critically useful for HFI—a eld that, to date, lacks
common standards. Our Lit Review app and taxonomy are
open-ended and inclusive mechanisms for researchers to ne-
gotiate their own understandings of HFI. Together, they af-
ford more democratic—and useful—sense-making than could
be achieved if we were to claim authority of our perspec-
tive on this remarkably heterogeneous eld. Our community
validation process supports this stance: researchers from di-
vergent backgrounds conrm the usefulness of the tool; they
used the model to make dierent sense of the HFI literature
and raise useful insights that assisted us to evolve the model.
While responses to our survey were minimal (11 from 129
authors), those who responded expressed enthusiasm for
the tool, saying that it responded to an important need. A
key factor of its success will be if researchers appropriate it,
and share leadership and input through representation on
the ’quality control’ committee. Moving forward, we expect
to organise inclusive events to discuss and rethink the app
and conceptual model, and—as a community—identify and
address upcoming challenges and opportunities emerging in
HFI. Our contribution can be considered a living, breathing
set of tools that can—and should—grow with the eld. In
doing so it will make visible evolving understandings of HFI.
Challenges Emerging in HFI
Using our tool and conceptual model, we examined the cur-
rent state of HFI research, mapping remarkably divergent
research concerns through a single conceptual model. In our
analysis, we identied a number of challenges the HFI com-
munity can respond to, e.g. a dominance of techno-solutionist
[
45
] approaches to HFI. In our dataset, contributions that
x, speed up, ease, or otherwise make interactions with food
more ecient, clearly outweigh those that explore the social,
playful, or cultural aspects of food practices. We identied
few interventions that support social bonding around food,
e.g. technology-mediated eating experiences that put the
focus beyond the individual (e.g. [
24
,
29
]). Yet, such aspects
are important to human relationships with (and through)
food [
65
]. Instrumental approaches to HFI produce invalu-
able knowledge on how to optimize our interactions with
food. However, we believe they could be balanced with other
approaches that examine the human-food-technology triad
from dierent lenses.
We also saw a dominance of technology-centric approaches
in source,produce and track papers. We suggest that enhanc-
ing user agency—using technology to support people to con-
duct food practices by themselves, rather than simply doing
the task for them—has value. Approaches such as [
14
,
24
,
61
],
for example, decrease the centrality of technology in the
interplay between humans and food, and lead to enriched
experiences. Critical reection on the notions of agency and
focus may be key to shaping the future of HFI. Such focus
could help to ensure that advances in technology do not
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 9
come at the cost of enriched, embodied engagement with
and through food.
Strength in Diversity
HFI encompasses a diversity of approaches, methodologies
and aims. While diversity is certainly an asset, it can also be
a barrier to communication and knowledge sharing between
researchers. Our dataset indicates little crossover between
the three communities we identied within HFI research.
We suggest this lack of crossover may impact research. The
rapid growth of HFI as a research area can make it di-
cult for researchers to keep track of changes in the eld,
let alone make sense of these changes—individually or as
a community. While conference workshops allow scholars
to gather, conferences happen once a year. There is some
transversal movement between conferences. Nonetheless,
there seems little to no crossover between the three HFI com-
munities, in part because researchers come from dierent
backgrounds, embrace dierent methodologies, and pursue
dierent outcomes. We believe these dierences can be a bar-
rier to establishing fruitful conversations about the future of
the eld as a whole. Our tool aims to address such barriers by
being openly accessible—online—and by encompassing the
rich plurality of perspectives within HFI. It is a lightweight
mechanism to support community sense-making of the eld
that aords negotiation in the analysis of HFI works, and
can therefore facilitate and strengthen community bonds.
It also enables researchers to contrast often divergent per-
spectives without having to meet. It thus facilitates on-going
mapping of the evolution of HFI, between conferences and
communities.
Responses to our tool suggest it could be useful beyond
HFI, to characterise elds of research that are similarly dy-
namic, heterogeneous and emergent. Our taxonomy and
tool aord analysis of divergent publications through a di-
versity of perspectives. They facilitate community-driven
sense-making of a research eld. The app enables diractive
reading of the data, and allows scholars from dierent disci-
plines to take divergent perspectives on the same dataset—to
identify gaps, critically engage with and position their re-
search concerns within the eld. Further, because the tool is
open and online, it is available to any interested researcher.
We believe our approach might help emerging researchers in
particular to track, shape and rethink the evolution of other
research elds that are as uid and diverse as HFI.
Our study demonstrates the potential of combining human
and computational skills. Computation aords visualization
of overwhelming datasets—it provided us with an overview
of HFI, and allowed us to lter the data to specic queries.
While we leveraged computation where possible, human
skill was key to our study. Collecting publications manually
allowed us to experience the nuances of the data rst hand,
because we had no choice but to engage with it deeply to
make sense of it. The resulting knowledge facilitated the
creation of our conceptual model—a taxonomy of the HFI
eld that responds to a diversity of perspectives. Embrac-
ing a qualitative and interpretive approach to data analysis
ensured an appropriately rich construction of the dataset.
Further, by involving the community in the interpretation
and classication of their research, we could ensure that
the dataset reected diering understandings of HFI. We
suggest that combining computational and human eorts
might be useful in other elds. Although it requires eort,
it aords rich, nuanced analysis of contributions. We argue
that such characteristics are important, in particular when
characterising emerging elds.
8 FUTURE WORK
For the HFI Lit Review App to be useful, the dataset must
be updated whenever there is a conference, event or jour-
nal issue that includes HFI-related publications. Further, to
ensure robust quality control that accurately represents the
community, the committee undertaking this task must have
a rotating membership that includes researchers from the
three communities identied.
During our study, a number of interface changes were
proposed for our tool. For example, displaying the number
of data points in an area selected by the user or enabling
zoom to explore clusters in detail. We intend to address these
changes. We also plan to implement an algorithm to scrape
publications automatically from known sources and to ex-
tend the tool to indicate new data points. Doing so will enable
us to sustainably complement human and computational
methods, and thereby ensure that the HFI database is robust
and reective of the diverse community’s concerns.
Participants also oered insights into the taxonomy. Some
changes have been implemented (e.g. allowing multiple do-
mains). Moving forward, we will consider expanding the
taxonomy by: rethinking the focus lens, for example to bet-
ter respond to contributions that are both functional and
socially-oriented; highlighting post-human approaches; or
implementing new domains, for example to enable categori-
sation of methodology-related contributions. To ensure rel-
evance, we will open up this process to other researchers
through community consultation.
Finally, we will develop our approach to mapping emerg-
ing elds of research by expanding our focus beyond HFI.
We have identied a number of researchers from diverse mul-
tidisciplinary elds (e.g. Animal-Computer Interaction, and
internet of worn things) who are interested to work with us,
to adapt the model to t their elds of interest. We anticipate
this expansion beyond HFI will lead to an enrichment of the
process we describe here.
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 10
9 CONCLUSION
We have presented a Systematic Mapping Study of Human-
Food Interaction research. To facilitate the mapping study,
we created a dataset of 270 HFI-related publications and de-
veloped a taxonomy to categorise them. We developed an
online data visualisation tool to aord diractive reading of
the data, and analysis from divergent perspectives. Using the
tool, we identied trends in the HFI research landscape. For
example, we saw a remarkable diversity of approaches, and
a dominance of instrumental approaches to the role of tech-
nology in food practice. These trends allow us to highlight
challenges and opportunities that HFI researchers may need
to address. We also identied a need for mechanisms—such
as our tool and taxonomy—that aord ongoing community
sense-making. Further, we identied the notions of agency
and focus as being particularly useful to reect on both the
state and potential of HFI research.
The trends, challenges and opportunities we present here
represent the current state of HFI—a state that, because of
the uid nature of the eld, we expect will evolve rapidly. We
encourage the community to revisit our analysis regularly,
using our tool, to co-create a continually evolving dataset
and reect on the interrelation of publications through varied
interpretive lenses. Our tool is lightweight and accessible,
and aords rich interaction between researchers. Its ongoing
usefulness depends on community interaction.
Our HFI Lit Review App leverages the potential of com-
putational tools to visualize a complex dataset and embraces
community-driven qualitative interpretation of data. The
tool is designed to aord diractive reading of the dataset,
allowing the reader to embrace divergent perspectives. Our
approach brings together quantitative and qualitative, com-
putational and human methods. We believe it can add value
beyond HFI, and that further research will help unpack that
value. We discussed how this approach may be useful in
elds that share HFI’s dynamic and heterogeneous nature.
Interaction with other researchers conrmed our impression
that our approach has relevance beyond HFI. To test this
potential, we intend to continue developing the tool, both
within and beyond HFI.
REFERENCES
[1]
2009. Proceedings of the ACM Multimedia 2009 Workshop on Multimedia
for Cooking and Eating Activities. ACM, New York, NY, USA.
[2]
2012. CEA ’12: Proceedings of the ACM Multimedia 2012 Workshop on
Multimedia for Cooking and Eating Activities. ACM, New York, NY,
USA. 433127.
[3]
2016. Proceedings of the 2nd International Workshop on Multimedia
Assisted Dietary Management. ACM, New York, NY, USA.
[4]
2018. DIS ’18: Proceedings of the 2018 Designing Interactive Systems
Conference. ACM, New York, NY, USA.
[5]
Ferran Altarriba Bertran, Samvid Jhaveri, Rosa Lutz, Katherine Isbister,
and Danielle Wilde. 2018. Visualising the Landscape of Human-Food
Interaction Research. In Proceedings of the 2018 ACM Conference Com-
panion Publication on Designing Interactive Systems (DIS ’18 Compan-
ion). ACM, New York, NY, USA, 243–248. https://doi.org/10.1145/
3197391.3205443
[6]
Ferran Altarriba Bertran and Danielle Wilde. 2018. Playing with food:
reconguring the gastronomic experience through play. In Proceedings
of the 1st International Conference on Food Design and Food Studies
(EFOOD 2017), October.
[7]
Judit Bar-Ilan and Noa Aharony. 2014. Twelve Years of Wikipedia
Research. In Proceedings of the 2014 ACM Conference on Web Science
(WebSci ’14). ACM, New York, NY, USA, 243–244. https://doi.org/10.
1145/2615569.2615643
[8]
Karen Barad. 2003. Posthumanist performativity: Toward an under-
standing of how matter comes to matter. Signs: Journal of women in
culture and society 28, 3 (2003), 801–831.
[9]
Karen Barad. 2007. Meeting the universe halfway: Quantum physics
and the entanglement of matter and meaning. duke university Press.
[10]
Balbir Barn, Souvik Barat, and Tony Clark. 2017. Conducting System-
atic Literature Reviews and Systematic Mapping Studies. In Proceedings
of the 10th Innovations in Software Engineering Conference (ISEC ’17).
ACM, New York, NY, USA, 212–213. https://doi.org/10.1145/3021460.
3021489
[11]
Fabian Beck, Michael Burch, Stephan Diehl, and Daniel Weiskopf. 2014.
The state of the art in visualizing dynamic graphs. EuroVis STAR 2
(2014), 1–21.
[12]
Jeanne Bloch and Celine Verchere. 2018. Using Art as an Insight to
Identify Ethical and Sustainable Issues. In Proceedings of the Designing
Recipes for Digital Food Futures, a CHI workshop.
[13]
Jaz Hee-jeong Choi, Marcus Foth, and Greg Hearn. 2014. Eat, cook,
grow: Mixing human-computer interactions with human-food interac-
tions. MIT Press.
[14]
Chia-Fang Chung, Elena Agapie, Jessica Schroeder, Sonali Mishra,
James Fogarty, and Sean A. Munson. 2017. When Personal Tracking
Becomes Social: Examining the Use of Instagram for Healthy Eating. In
Proceedings of the 2017 CHI Conference on Human Factors in Computing
Systems (CHI ’17). ACM, New York, NY, USA, 1674–1687. https://doi.
org/10.1145/3025453.3025747
[15]
Simon Clatworthy, Robin Oorschot, and Berit Lindquister. 2014. How
to Get a Leader to Talk: Tangible Objects for Strategic Conversations in
Service Design. In ServDes. 2014 Service Future; Proceedings of the fourth
Service Design and Service Innovation Conference; Lancaster University;
United Kingdom; 9-11 April 2014. Linköping University Electronic Press,
270–280.
[16]
Manolo J Cobo, Antonio Gabriel López-Herrera, Enrique Herrera-
Viedma, and Francisco Herrera. 2011. Science mapping software tools:
Review, analysis, and cooperative study among tools. Journal of the
American Society for Information Science and Technology 62, 7 (2011),
1382–1402.
[17]
Rob Comber, Jaz Hee-Jeong Choi, Jettie Hoonhout, and Kenton O’hara.
2014. Designing for human–food interaction: an introduction to the
special issue on ‘food and interaction design’. International Journal of
Human-Computer Studies 72, 2 (2014), 181–184.
[18]
Rob Comber, Eva Ganglbauer, Jaz Hee-jeong Choi, Jettie Hoonhout,
Yvonne Rogers, Kenton O’Hara, and Julie Maitland. 2012. Food and
Interaction Design: Designing for Food in Everyday Life. In CHI ’12
Extended Abstracts on Human Factors in Computing Systems (CHI EA
’12). ACM, New York, NY, USA, 2767–2770. https://doi.org/10.1145/
2212776.2212716
[19]
Felicia Cordeiro, Daniel A. Epstein, Edison Thomaz, Elizabeth Bales,
Arvind K. Jagannathan, Gregory D. Abowd, and James Fogarty. 2015.
Barriers and Negative Nudges: Exploring Challenges in Food Jour-
naling. In Proceedings of the 33rd Annual ACM Conference on Human
Factors in Computing Systems (CHI ’15). ACM, New York, NY, USA,
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 11
1159–1162. https://doi.org/10.1145/2702123.2702155
[20]
Amélie Cordier and Emmanuel Nauer. 2012. Proceedings of the 2012
Cooking with Computers workshop. In Proceedings of the European
Conference on Articial Intelligence (ECAI) 2012.
[21]
Amélie Cordier, Emmanuel Nauer, and Michael Wiegand. 2013. Pro-
ceedings of the 2013 Cooking with Computers workshop. In Proceed-
ings of the International Joint Conferences on Articial Intelligence (IJ-
CAI) 2013.
[22]
Zack Denfeld. 2017. Food Phreaking. In Proceedings of the 2017 ACM
SIGCHI Conference on Creativity and Cognition (C&C ’17). ACM, New
York, NY, USA, 466–468. https://doi.org/10.1145/3059454.3073726
[23]
Carl DiSalvo and Tom Jenkins. 2017. Fruit Are Heavy: A Prototype
Public IoT System to Support Urban Foraging. In Proceedings of the
2017 Conference on Designing Interactive Systems (DIS ’17). ACM, New
York, NY, USA, 541–553. https://doi.org/10.1145/3064663.3064748
[24]
Markéta Dolejšová and Cindy Lin Kaiying. 2016. Squat & grow: De-
signing smart human-food interactions in Singapore. In Proceedings
of the SEACHI 2016 on Smart Cities for Better Living with HCI and UX.
ACM, 24–27.
[25]
Markéta Dolejšová and Denisa Kera. 2017. The Fermentation GutHub
Project and the Internet of Microbes. In Enriching Urban Spaces with
Ambient Computing, the Internet of Things, and Smart City Design. IGI
Global, 25–46.
[26]
Markéta Dolejšová, Rohit Ashok Khot, Hilary Davis, Hasan Shahid
Ferdous, and Andrew Quitmeyer. 2018. Designing Recipes for Digital
Food Futures. In Extended Abstracts of the 2018 CHI Conference on
Human Factors in Computing Systems (CHI EA ’18). ACM, New York,N Y,
USA, Article W10, 8 pages. https://doi.org/10.1145/3170427.3170622
[27]
Geremy Farr-Wharton, Jaz Hee-Jeong Choi, and Marcus Foth. 2014.
Technicolouring the Fridge: Reducing Food Waste Through Uses of
Colour-coding and Cameras. In Proceedings of the 13th International
Conference on Mobile and Ubiquitous Multimedia (MUM ’14). ACM,
New York, NY, USA, 48–57. https://doi.org/10.1145/2677972.2677990
[28]
Katia Romero Felizardo, Elisa Yumi Nakagawa, Stephen G. MacDonell,
and José Carlos Maldonado. 2014. A Visual Analysis Approach to
Update Systematic Reviews. In Proceedings of the 18th International
Conference on Evaluation and Assessment in Software Engineering (EASE
’14). ACM, New York, NY, USA, Article 4, 10 pages. https://doi.org/10.
1145/2601248.2601252
[29]
Hasan Shahid Ferdous, Frank Vetere, Hilary Davis, Bernd Ploderer,
Kenton O’Hara, Rob Comber, and Geremy Farr-Wharton. 2017. Cele-
bratory Technology to Orchestrate the Sharing of Devices and Stories
During Family Mealtimes. In Proceedings of the 2017 CHI Conference
on Human Factors in Computing Systems (CHI ’17). ACM, New York,
NY, USA, 6960–6972. https://doi.org/10.1145/3025453.3025492
[30]
Greg Guest, Kathleen M MacQueen, and Emily E Namey. 2011. Applied
thematic analysis. sage.
[31]
Kim Halskov and Nicolai Brodersen Hansen. 2015. The diversity of
participatory design research practice at PDC 2002–2012. International
Journal of Human-Computer Studies 74 (2015), 81–92.
[32]
Donna Haraway. 1992. The promises ofmonsters: a regenerative poli-
tics for inappropriate/d others. Cultural studies (1992), 295–337.
[33]
Steve Harrison, Deborah Tatar, and Phoebe Sengers. 2007. The three
paradigms of HCI. In Alt. Chi. Session at the SIGCHI Conference on
Human Factors in Computing Systems San Jose, California, USA. 1–18.
[34]
Atsushi Hashimoto, Jin Inoue, Kazuaki Nakamura, Takuya Funatomi,
Mayumi Ueda, Yoko Yamakata, and Michihiko Minoh. 2012. Recogniz-
ing Ingredients at Cutting Process by Integrating Multimodal Features.
In Proceedings of the ACM Multimedia 2012 Workshop on Multimedia
for Cooking and Eating Activities (CEA ’12). ACM, New York, NY, USA,
13–18. https://doi.org/10.1145/2390776.2390780
[35]
Tad Hirsh. 2014. Alleys to Appetizers: Taking a Systems Approach
to Urban Agriculture. In Eat, cook, grow: Mixing human-computer
interactions with human-food interactions, Jaz Hee-jeong Choi, Marcus
Foth, and Greg Hearn (Eds.). MIT press.
[36]
Ichiro Ide and Yoko Yamakata. 2017. Proceedings of the 9th Workshop
on Multimedia for Cooking and Eating Activities. In Proceedings of the
2017 International Joint Conference on Articial Intelligence.
[37]
Azusa Kadomura, Cheng-Yuan Li, Koji Tsukada, Hao-Hua Chu, and
Itiro Siio. 2014. Persuasive Technology to Improve Eating Behav-
ior Using a Sensor-embedded Fork. In Proceedings of the 2014 ACM
International Joint Conference on Pervasive and Ubiquitous Comput-
ing (UbiComp ’14). ACM, New York, NY, USA, 319–329. https:
//doi.org/10.1145/2632048.2632093
[38]
Victor Kaptelinin and Bonnie Nardi. 2012. Activity theory in HCI:
Fundamentals and reections. Synthesis Lectures Human-Centered
Informatics 5, 1 (2012), 1–105.
[39]
Rohit Ashok Khot, Deepti Aggarwal, Ryan Pennings, Larissa Hjorth,
and Florian ’Floyd’ Mueller. 2017. EdiPulse: Investigating a Playful
Approach to Self-monitoring Through 3D Printed Chocolate Treats. In
Proceedings of the 2017 CHI Conference on Human Factors in Computing
Systems (CHI ’17). ACM, New York, NY, USA, 6593–6607. https://doi.
org/10.1145/3025453.3025980
[40]
Bongshin Lee, Mary Czerwinski, George Robertson, and Benjamin B.
Bederson. 2005. Understanding Research Trends in Conferences Using
paperLens. In CHI ’05 Extended Abstracts on Human Factors in Com-
puting Systems (CHI EA ’05). ACM, New York, NY, USA, 1969–1972.
https://doi.org/10.1145/1056808.1057069
[41]
Peter Lyle, Jaz Hee-jeong Choi, and Marcus Foth. 2015. Growing
Food in the City: Design Ideations for Urban Residential Gardeners.
In Proceedings of the 7th International Conference on Communities and
Technologies (C&T ’15). ACM, New York, NY, USA, 89–97. https:
//doi.org/10.1145/2768545.2768549
[42]
Bruno Mesz, Kevin Herzog, Juan Cruz Amusategui, Lucas Samaruga,
and Sebastián Tedesco. 2017. Let’s Drink This Song Together: Inter-
active Taste-sound Systems. In Proceedings of the 2Nd ACM SIGCHI
International Workshop on Multisensory Approaches to Human-Food
Interaction (MHFI 2017). ACM, New York, NY, USA, 13–17. https:
//doi.org/10.1145/3141788.3141791
[43]
Robb Mitchell, Alexandra Papadimitriou, Youran You, and Laurens
Boer. 2015. Really Eating Together: A Kinetic Table to Synchronise
Social Dining Experiences. In Proceedings of the 6th Augmented Human
International Conference (AH ’15). ACM, New York, NY, USA, 173–174.
https://doi.org/10.1145/2735711.2735822
[44]
Shinsuke Mori, Tetsuro Sasada, Yoko Yamakata, and Koichiro Yoshino.
2012. A Machine Learning Approach to Recipe Text Processing. In
Proceedings of the 2012 Cooking with Computers (CwC) workshop.
[45]
Evgeny Morozov. 2013. To save everything, click here: The folly of
technological solutionism. Public Aairs.
[46]
Hiromi Nakamura and Homei Miyashita. 2013. Controlling Saltiness
Without Salt: Evaluation of Taste Change by Applying and Releasing
Cathodal Current. In Proceedings of the 5th International Workshop on
Multimedia for Cooking & Eating Activities (CEA ’13). ACM, New
York, NY, USA, 9–14. https://doi.org/10.1145/2506023.2506026
[47]
Vladimir Nedovic. 2013. Learning ingredient space with generative
probabilistic models. In Proceedings of the 2013 Cooking with Computers
(CwC) workshop.
[48]
Truong-Huy D Nguyen, Edward Melcer, Alessandro Canossa, Kather-
ine Isbister, and Magy Seif El-Nasr. 2018. Seagull: A birdâĂŹs-eye view
of the evolution of technical games research. Entertainment computing
26 (2018), 88–104.
[49]
Paavo Nieminen, Ilkka Pölönen, and Tuomo Sipola. 2013. Research
literature clustering using diusion maps. Journal of Informetrics 7, 4
(2013), 874–886.
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 12
[50]
Antinus Nijholt, Carlos Velasco, Gijs Huisman, and Kasun
Karunanayaka. 2016. Proceedings of the 1st Workshop on Multi-
Sensorial Approaches to Human-Food Interaction. ACM.
[51]
Marianna Obrist. 2017. Mastering the Senses in HCI: Towards Mul-
tisensory Interfaces. In Proceedings of the 12th Biannual Conference
on Italian SIGCHI Chapter (CHItaly ’17). ACM, New York, NY, USA,
Article 2, 2 pages. https://doi.org/10.1145/3125571.3125603
[52]
Doenja Oogjes, Miguel Bruns, and Ron Wakkary. 2016. Lyssna: A
Design Fiction to Reframe Food Waste. In Proceedings of the 2016 ACM
Conference Companion Publication on Designing Interactive Systems
(DIS ’16 Companion). ACM, New York, NY, USA, 109–112. https:
//doi.org/10.1145/2908805.2909401
[53]
Antoine Ponsard, Francisco Escalona, and Tamara Munzner. 2016.
PaperQuest: A Visualization Tool to Support Literature Review. In
Proceedings of the 2016 CHI Conference Extended Abstracts on Human
Factors in Computing Systems (CHI EA ’16). ACM, New York, NY, USA,
2264–2271. https://doi.org/10.1145/2851581.2892334
[54]
Sebastian Prost, Clara Crivellaro, Andy Haddon, and Rob Comber.
2018. Food Democracy in the Making: Designing with Local Food
Networks. In Proceedings of the 2018 CHI Conference on Human Factors
in Computing Systems (CHI ’18). ACM, New York, NY, USA, Article
333, 14 pages. https://doi.org/10.1145/3173574.3173907
[55]
Hans Rosling. [n. d.]. Let my dataset change your mindset. https:
//www.ted.com/talks/hans_rosling_at_state
[56]
Monika Rut and Markéta Dolejšová. 2018. Digital Food Sharing Prac-
tices and Controversies. In Proceedings of the Designing Recipes for
Digital Food Futures, a CHI workshop.
[57]
Andreas Seiderer, Simon Flutura, and Elisabeth André. 2017. Develop-
ment of a mobile multi-device nutrition logger. In Proceedings of the
2nd ACM SIGCHI International Workshop on Multisensory Approaches
to Human-Food Interaction. ACM, 5–12.
[58]
Charles Spence. 2017. Gastrophysics: the new science of eating. Penguin
UK.
[59]
Charles Spence, Marianna Obrist, Carlos Velasco, and Nimesha Ranas-
inghe. 2017. Digitizing the chemical senses: possibilities & pitfalls.
International Journal of Human-Computer Studies 107 (2017), 62–74.
[60]
Hu Tao, Mark A Brenckle, Miaomiao Yang, Jingdi Zhang, Mengkun
Liu, Sean M Siebert, Richard D Averitt, Manu S Mannoor, Michael C
McAlpine, John A Rogers, et al
.
2012. Silk-based conformal, adhesive,
edible food sensors. Advanced Materials 24, 8 (2012), 1067–1072.
[61]
Erica Vannucci, Ferran Altarriba, Justin Marshall, and Danielle Wilde.
2018. Handmaking Food Ideals: Crafting the Design of Future Food-
related Technologies. In Proceedings of the 2018 ACM Conference Com-
panion Publication on Designing Interactive Systems (DIS ’18 Compan-
ion). ACM, New York, NY, USA, 419–422. https://doi.org/10.1145/
3197391.3197403
[62]
Carlos Velasco, Felipe Reinoso Carvalho, Olivia Petit, and Anton Ni-
jholt. 2016. A multisensory approach for the design of food and drink
enhancing sonic systems. In Proceedings of the 1st Workshop on Multi-
sensorial Approaches to Human-Food Interaction. ACM, 7.
[63]
Carlos Velasco, Anton Nijholt, Marianna Obrist, Katsunori Okajima,
Rick Schierstein, and Charles Spence. 2017. MHFI 2017: 2Nd In-
ternational Workshop on Multisensorial Approaches to Human-food
Interaction (Workshop Summary). In Proceedings of the 19th ACM Inter-
national Conference on Multimodal Interaction (ICMI 2017). ACM, New
York, NY, USA, 674–676. https://doi.org/10.1145/3136755.3137023
[64]
Juergen Wagner, Gijs Geleijnse, and Aart van Halteren. 2011. Guidance
and support for healthy food preparation in an augmented kitchen.
In Proceedings of the 2011 Workshop on Context-awareness in Retrieval
and Recommendation. ACM, 47–50.
[65]
Danielle Wilde and Ferran Altarriba Bertran. in press. From Playing
with Food to Participatory Research through Gastronomy Design:
a designerly move towards more playful gastronomy. International
Journal of Food Design 4 (in press).
[66]
Longqi Yang, Cheng-Kang Hsieh, Hongjian Yang, John P Pollak, Nicola
Dell, Serge Belongie, Curtis Cole, and Deborah Estrin. 2017. Yum-
me: a personalized nutrient-based meal recommender system. ACM
Transactions on Information Systems (TOIS) 36, 1 (2017), 7.
[67]
Xu Ye, Guanling Chen, Yang Gao, Honghao Wang, and Yu Cao. 2016.
Assisting Food Journaling with Automatic Eating Detection. In Proceed-
ings of the 2016 CHI Conference Extended Abstracts on Human Factors in
Computing Systems (CHI EA ’16). ACM, New York, NY, USA, 3255–3262.
https://doi.org/10.1145/2851581.2892426
[68]
Rui Zhang and Oliver Amft. 2016. Regular-look Eyeglasses Can Moni-
tor Chewing. In Proceedings of the 2016 ACM International Joint Con-
ference on Pervasive and Ubiquitous Computing: Adjunct (UbiComp ’16).
ACM, New York, NY, USA, 389–392. https://doi.org/10.1145/2968219.
2971374
[69]
Amit Zoran and Dror Cohen. 2018. Digital Konditorei: Programmable
Taste Structures Using a Modular Mold. In Proceedings of the 2018 CHI
Conference on Human Factors in Computing Systems (CHI ’18). ACM,
New York, NY, USA, Article 400, 9 pages. https://doi.org/10.1145/
3173574.3173974
CHI 2019 Paper
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
Paper 678
Page 13
... Celebratory technology involves, for example, individuals who use cooking as a creative way "to imagine, for example, technologies that support them in adapting recipes to fit their personal tastes and personalities and applications that help them explore new flavors and cuisines" [11]. Further, Altarriba Bertran et al. [2] show the connection between food, taste, and socio-cultural circumstances. The latter could, for example, lead to food neophobia [25], which means that consumers have a significant aversion to trying unknown food products. ...
... Moreover, automatic diet tracking systems [3] and personal food computers for urban agriculture [9] that support personalized nutrition and home farming practices. This "Human-Food Interaction (HFI)" work [5,14,17,25,46,47] has become a major part of HCI research into "the interconnection between the self and food" [13,14]. ...
Conference Paper
Recognizing the significant potential impact that HCI has on food practices and experiences, researchers and practitioners are undertaking a growing number of explorations of novel computing technology and food combinations. These explorations have so far primarily emphasized technology-driven systems and taken a human-centric perspective. We propose a Special Interest Group (SIG) in "foodHCI futures"that creates a space for researchers to discuss the boundaries of food incorporating HCI, and with the simultaneous aims of reconciling food with technology and extending our visions for human-food interactions towards anthropocentrism. Specifically, the SIG will be a beginning of developing a structured conceptual map of the possibilities for future technology interventions in food systems. In developing this map, we hope to encourage democratized debate, provoke new and divergent thoughts on the opportunities for foodHCI, and ultimately gain unique insights that contribute to preferable food futures.
Article
Full-text available
Previous research shows that eating together (i.e., commensality) impacts food choice, time spent eating, and enjoyment. Conversely, eating alone is considered a possible cause of unhappiness. In this paper, we conceptually explore how interactive technology might allow for the creation of artificial commensal companions: embodied agents providing company to humans during meals (e.g., a person living in isolation due to health reasons). We operationalize this with the design of our commensal companion: a system based on the MyKeepon robot, paired with a Kinect sensor, able to track the human commensal's activity (i.e., food picking and intake) and able to perform predefined nonverbal behavior in response. In this preliminary study with 10 participants, we investigate whether this autonomous social robot-based system can positively establish an interaction that humans perceive and whether it can influence their food choices. In this study, the participants are asked to taste some chocolates with and without the presence of an artificial commensal companion. The participants are made to believe that the study targets the food experience, whilst the presence of a robot is accidental. Next, we analyze their food choices and feedback regarding the role and social presence of the artificial commensal during the task performance. We conclude the paper by discussing the lessons we learned about the first interactions we observed between a human and a social robot in a commensality setting and by proposing future steps and more complex applications for this novel kind of technology.
Book
Full-text available
Eating and drinking are undoubtedly amongst life’s most multisensory experiences. Take, for instance, the enjoyment of flavor, which is one of the most important elements of such experiences, resulting from the integration of gustatory, (retronasal) olfactory, and possibly also trigeminal/oral-somatosensory cues. Nevertheless, researchers have suggested that all our senses can influence the way in which we perceive flavor, not to mention our eating and drinking experiences. For instance, the color and shape of the food, the background sonic/noise cues in our eating environments, and/or the sounds associated with mastication can all influence our perception and enjoyment of our eating and drinking experiences. Human-Food Interaction (HFI) research has been growing steadily in recent years. Research into multisensory interactions designed to create, modify, and/or enhance our food-related experiences is one of the core areas of HFI (Multisensory HFI or MHFI). The aim being to further our understanding of the principles that govern the systematic connections between the senses in the context of HFI. In this Research Topic, we called for investigations and applications of systems that create new, or enhance already existing, multisensory eating and drinking experiences (what can be considered the “hacking” of food experiences) in the context of HFI. Moreover, we were also interested in those works that focus on or are based on the principles governing the systematic connections that exist between the senses. HFI also involves the experiencing of food interactions digitally in remote locations. Therefore, we were also interested in sensing and actuation interfaces, new communication mediums, and persisting and retrieving technologies for human food interactions. Enhancing social interactions to augment the eating experience is another issue we wanted to see addressed here, what has been referred to as “digital commensality”.
Article
Full-text available
Eating and drinking are undoubtedly amongst life’s most multisensory experiences. Take, for instance, the enjoyment of flavor, which is one of the most important elements of such experiences, resulting from the integration of gustatory, (retronasal) olfactory, and possibly also trigeminal/oral-somatosensory cues (Prescott, 2015). Nevertheless, researchers have suggested that all our senses can influence the way in which we perceive flavor, not to mention our eating and drinking experiences. For instance, the color and shape of the food, the background sonic/noise cues in our eating environments, and/or the sounds associated with mastication can all influence our perception and enjoyment of our eating and drinking experiences (Spence, 2020). Human-Food Interaction (HFI) research has been growing steadily in recent years (e.g., Deng et al., 2021). Research into multisensory interactions designed to create, modify, and/or enhance our food-related experiences is one of the core areas of HFI (Multisensory HFI or MHFI, Altarriba Bertran et al., 2019; Velasco and Obrist, 2020). The aim being to further our understanding of the principles that govern the systematic connections between the senses in the context of HFI. In this Research Topic, we called for investigations and applications of systems that create new, or enhance already existing, multisensory eating and drinking experiences (what can be considered the “hacking” of food experiences) in the context of HFI. Moreover, we were also interested in those works that focus on or are based on the principles governing the systematic connections that exist between the senses. HFI also involves the experiencing of food interactions digitally in remote locations. Therefore, we were also interested in sensing and actuation interfaces, new communication mediums, and persisting and retrieving technologies for human food interactions. Enhancing social interactions to augment the eating experience is another issue we wanted to see addressed here, what has been referred to as “digital commensality” (Spence et al., 2019).
Article
Food and nutrition are important to ensuring health, and practices related to food (including obtaining, preparing, consuming, sharing and cleaning up food) involve effortful activity - work. Human Factors and Ergonomics (HF/E) efforts that leverage expertise in across HF/E domains (i.e., physical, cognitive and organizational ergonomics) can provide a holistic approach to supporting human wellbeing and performance in these food practices. For example, ergonomists can attend to the physical environment and tools, the mental processes and/or overall sociotechnical systems involved in food practice all to facilitate food practices that increase the health and wellbeing of humans across diverse needs, abilities and limitations. This panel will introduce the ergonomics of food, highlighting ongoing work as well as opportunities for further work impact.
Conference Paper
Full-text available
Much technology is designed to help people enact processes faster and more precisely. Yet, these advantages can come at the cost of other, perhaps less tangible, values. In this workshop we aim to articulate values associated with handmade through a co-creative exploration in the food domain. Our objective is to explore the potential of integrating such values into future food-related technologies. In a full day workshop we will: critically reflect on the notion of handmade; engage actively with food-production, plating and consumption-as design material; and conduct collective discussions around the values that these processes and materials can embody when attended to through lenses other than efficiency. By handmaking: touching, smelling, tasting, listening, speaking and enacting choreographies with the materials at hand, we hope to deepen the discussion of the meaning associated with the handmade and bring a richness to ways that designers imagine future food-related technologies.
Conference Paper
Full-text available
While conducting a review of food-related technology research, we discovered that activity in this area is skyrocketing across a broad range of disciplinary interests and concerns. The dynamic and heterogeneous nature of the research presents a challenge to scholars wishing to critically engage with prior work, identify gaps and ensure impact. In response to this challenge, we are developing an online visualisation tool: an app that affords diffractive reading of the literature, mapping interferences and differences from varied perspectives. We present our first iteration of the app, which enables scholars to navigate the literature through seven lenses-focus, agency, domain, date of publication, author keywords, and publication venue and type. Here we present the first iteration of the app, toward receiving critical input from concerned researchers, to validate our approach and ensure relevance moving forward.
Conference Paper
Full-text available
Digital food technologies such as diet trackers, food sharing apps, and 'smart' kitchenware offer promising yet debatable food futures. While proponents suggest its potential to prompt efficient food lifestyles, critics highlight the underlying technosolutionism of digital food innovation and limitations related to health safety and data privacy. This workshop addresses both present and near-future digital food controversies and seeks to extend the existing body of Human-Food Interaction (HFI) research. Through scenarios and food-tech prototyping navigated by bespoke Digital Food Cards, we will unpack issues and suggest possible design approaches. We invite proposals from researchers, designers, and other practitioners interested in working towards a complex framework for future HFI research.
Conference Paper
Full-text available
This paper introduces the concept of 'food democracy' as a theoretical framing for HCI to engage in human-food interaction. Extending existing foci of health and environmental sustainability, food democracy requires thinking through aspects of social and economic justice, and democratic governance as directions for the study and design of technologies for alternative food movements. To exemplify food democracy, we report on field observations and interviews about the opportunities and challenges for supporting the development of local food networks with communities in deprived neighbourhoods using an online direct food marketing platform. Using a food democracy framing, we identify tensions around environmental, social, and economic goals; challenges of local food businesses operating within the existing economic paradigm; and differing perspectives on ownership and governance in the network. We discuss the need for HCI to design for systems change and propose a design space for HCI in supporting food democracy movements.
Conference Paper
Full-text available
We present an introduction paper to the second version of the workshop on ‘Multisensory Approaches to Human-Food Interaction’ to be held at the 19th ACM International Conference on Multimodal Interaction, which will take place on November 13th, 2017, in Glasgow, Scotland. Here, we describe the workshop’s objectives, the key contributions of the different position papers, and the relevance of their respective topic(s). Both multisensory research and new technologies are evolving fast which has opened up a number of possibilities for designing new ways of interacting with foods and drinks. This workshop highlights the rapidly growing interest in the field of Multisensory Human-Food Interaction, which can, for example, be observed in the variety of novel research and technology developments in the area.
Article
Socialization, eating and play are core activities that make us human. While they are often brought together, play theory suggests that their combination has unexplored potential in the context of gastronomy. Our research also indicates that a chef’s desire to control the meal may be a key impediment to developing dining experiences in which the diner’s playful engagement impacts taste, texture and flavour combination. We investigate if combining participatory research through design and play theory might better situate chefs to diversify their approach to playful gastronomy. Using experimental design methods, we interviewed a chef, a maître d’, a professional gastronomist, two food enthusiasts and a novice, to identify overlooked opportunities to extend play in gastronomy. We then conducted a series of dinners – designed with and for experts, enthusiasts and novices – to explore these opportunities, and tested the resulting method through a workshop with student chefs and game designers. We present the method: Participatory Research through Gastronomy Design (PRGD), using the case of its development to explicate its characteristics. Our research suggests that PRGD supports the design of playful gastronomic experiences that appeal to a range of diners, affords exploration of play’s impact on social dynamics and can productively inform concrete design choices. It also – crucially – supports chefs to partially transfer control of how a meal unfolds, without diluting their sense of controlling the overall experience. PRGD thus addresses a key impediment to extending play in gastronomy. Gastronomy that responds to diners’ needs and desires for play are currently limited. We propose PRGD as an exciting – and viable – approach to address this limitation.
Conference Paper
Digital Gastronomy (DG) is a culinary concept that enhances traditional cooking with new HCI capabilities, rather than replacing the chef with an autonomous machine. Preliminary projects demonstrate implementation of DG via the deployment of digital instruments in a kitchen. Here we contribute an alternative solution, demonstrating the use of a modular (silicone) mold and a genetic mold-arrangement algorithm to achieve a variety of shape permutations for a recipe, allowing the control of taste structures in the dish. The mold overcomes the slow production time of 3D food printing, while allowing for a high degree of flexibility in the numerous shapes produced. This flexibility enables us to satisfy chefs' and diners' diverse requirements. We present the mold's logic, arithmetic, design and special parts, the evolutionary algorithm, and a recipe, exploiting a new digital cooking concept of programmable edible taste structures and taste patterns to enrich user interaction with a given recipe.
Article
Within Entertainment Computing, games research has grown to be its own area, with numerous publication venues dedicated to it. As this area evolves, it is fruitful to examine its overall development—which subcommunities and research interests were present from the start, which have come and gone, and which are currently active—to better understand the research community as a whole and where it may proceed. In this paper, we present a data-driven analysis and interactive visualization tool to shed light on how technical domains within the games research field have evolved from 2000 - 2013, based on publication data from over 8,000 articles collected from 48 games research venues, including Entertainment Computing, FDG, AIIDE, and DiGRA. The approach we present is descriptive. We first used data mining algorithms to group related papers into clusters of similar research topics and evolve these clusters over time. We then designed an interactive visualization system, named Seagull, comprised of Sankey diagrams that allow us to interactively visualize and examine the transition and coalescing of different clusters across time. We present our descriptive analysis in this paper and also contribute the visualization interface to allow other researchers to examine the data and develop their own analysis.