Content uploaded by Ulrich Gnewuch
Author content
All content in this area was uploaded by Ulrich Gnewuch on May 18, 2018
Content may be subject to copyright.
This is the author’s version of a work that was published in the following source
Gnewuch, U., Morana, S., Heckmann, C., and Maedche, A. 2018. “Designing Conversational Agents for
Energy Feedback,” in Proceedings of the 13th International Conference on Design Science Research in
Information Systems and Technology (DESRIST 2018), Chennai, India, June 3–6.
Please note: Copyright is owned by the author and / or the publisher.
Commercial use is not allowed.
Institute of Information Systems and Marketing (IISM)
Fritz-Erler-Strasse 23
76133 Karlsruhe - Germany
http://iism.kit.edu
Karlsruhe Service Research Institute (KSRI)
Kaiserstraße 89
76133 Karlsruhe – Germany
http://ksri.kit.edu
© 2017. This manuscript version is made available under the CC-
BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-
nc-nd/4.0/
Designing Conversational Agents for Energy Feedback
Ulrich Gnewuch1,2 (), Stefan Morana1, Carl Heckmann2, and Alexander Maedche1
1 Institute of Information Systems and Marketing (IISM), Karlsruhe Institute of Technology
(KIT), Karlsruhe, Germany
{ulrich.gnewuch, stefan.morana, alexander.maedche}@kit.edu
2 hsag Heidelberger Service AG, Heidelberg, Germany
{u.gnewuch, c.heckmann}@hsag.info
Abstract. Reducing and shifting energy consumption could contribute signifi-
cantly to a more sustainable use of energy in households. Studies have shown
that the provision of feedback can encourage consumers to use energy more sus-
tainably. While there is wide variety of energy feedback solutions ranging from
in-home displays to mobile applications, there is a lack of research on whether
and how conversational agents can provide energy feedback to promote sustain-
able energy use. As conversational agents, such as chatbots, promise a natural
and intuitive user interface, they may have great potential for energy feedback.
This paper explores how to design conversational agents for energy feedback and
proposes design principles based on existing literature. The design principles are
instantiated in a text-based conversational agent and evaluated in a focus group
session with industry experts. We contribute with valuable design knowledge that
extends previous research on the design of energy feedback solutions.
Keywords: Conversational Agent, Chatbot, Energy Feedback, Focus Group,
Design Science Research.
1 Introduction
To combat climate change and reduce greenhouse gas emissions, significant invest-
ments are being made in new low-carbon technologies, renewable energy, energy effi-
ciency, and grid infrastructure [1]. To achieve the European Union’s (EU) ambitious
climate goals of reducing greenhouse gas emissions by 80% by 2050, all sectors are
expected to play their part [2]. Since energy consumption of the residential sector still
accounts for around 25% of total energy use in the EU [2], sustainable use of energy in
households could significantly contribute to reaching the EU’s climate goals. Sustain-
able energy use includes not only reducing energy consumption, but also shifting en-
ergy consumption to times when renewable energy sources (e.g., wind or solar power)
are abundant [3]. Particularly, non-time critical energy use in households, such as wash-
ing machines or dish washers, can be shifted away from peak demand periods [3].
Providing energy feedback to consumers has been found to increase energy use
awareness and promote both reducing and shifting energy consumption in households
[e.g., 3, 4]. Moreover, reviews of energy feedback research have found that the provi-
sion of feedback can result in average energy savings of 10% [5]. In the past, many
energy feedback solutions have been developed such as in-home displays, mobile ap-
plications, or web portals [5]. Given the large-scale deployment of smart meters that
collect high-frequency consumption data and the advances in algorithms for energy
disaggregation, these solutions are able to provide direct, real-time feedback on the
level of individual appliances [6]. Additionally, they can provide interactive feedback
augmented with additional approaches (e.g., personalized recommendations) to provide
greater opportunities to engage consumers over time [4]. Although much research has
been conducted on their design, recent reviews of energy feedback solutions indicate
that no research has been conducted on how conversational agents (CAs) can be used
to provide energy feedback [5, 7]. CAs, such as text-based chatbots or voice-based per-
sonal assistants like Amazon’s Alexa, promise a convenient and intuitive user interface
to interact with technology using natural language (i.e., written or spoken) [8]. Because
of advances in artificial intelligence and natural language processing, the capabilities
of CAs have improved significantly in recent years [9]. While they are limited in their
ability to provide visual information on energy consumption (e.g., in the form of graphs
or dashboards), they can leverage natural language to answer questions and provide
personalized feedback. Given the rising interest in CAs [10], we argue that there is an
opportunity to investigate the design of CAs for energy feedback to address the lack of
consumer awareness of energy use and facilitate a more sustainable use of energy in
households. Although feedback solutions using SMS or email have been developed for
related contexts [e.g., 11], research on how to design CAs for energy feedback is scarce.
Thus, we aim to fill this gap and explore the following research question:
How to design conversational agents for energy feedback to promote sustainable
use of energy in households?
To address this research question, we follow the design science research (DSR) [12]
approach to iteratively design and evaluate a text-based CA for energy feedback. Based
on a literature review on existing energy feedback solutions, we propose four design
principles for CAs for energy feedback. These principles are instantiated in a text-based
CA and evaluated in an exploratory focus group session [13] with domain experts from
the energy industry. The remainder of this paper is organized as follows. Section two
introduces related work on energy feedback and CAs. Section three outlines our DSR
project, while section four describes the proposed design of our artifact. In section five,
we present and discuss the findings of our evaluation, before we conclude the paper
with a short summary and a discussion of limitations in section six.
2 Related Work
2.1 Promoting Sustainable Energy Use through Feedback
Research in psychology has extensively studied feedback and its impact on behavior
change (for an overview, see [14]). Feedback is commonly understood as “the process
of giving people information about their behavior that can be used to reinforce and/or
modify future actions” [4]. In the context of energy use, feedback has been identified
as an effective intervention to promote sustainable use of energy (for a detailed review,
see [4, 15]). In general, energy feedback can be provided in different ways. Direct feed-
back is available in real-time, whereas indirect feedback is provided after the consump-
tion occurs [4]. Moreover, feedback can be aggregated (i.e., a household’s total energy
consumption) or on appliance-level [4]. Appliance-level feedback contains information
about individual devices, such as electronics or water heaters [6]. Furthermore, feed-
back can be combined with other interventions, such as goal setting or financial incen-
tives, to increase its effectiveness [4]. Reviews of energy feedback research have found
that the provision of feedback can result in average energy savings of 10% as well as
promote load shifting [5], but its effectiveness depends on the way it is provided [4].
Different technologies have been used to provide energy feedback such as in-home
displays, web portals, or mobile applications (for a detailed review of different solu-
tions, see [5]). Many of these energy feedback solutions visualize household energy
consumption based on data collected by sensors or smart meters [16], while others focus
their feedback on a single device such as a washing machine [e.g., 3]. Moreover, they
usually push out information to consumers (e.g., monthly energy reports) or require
consumers to pull information from them (e.g., web portal or mobile app) [4, 17]. Mod-
ern solutions also frequently include additional features such as community platforms
[18] or individual/social level comparisons [17]. However, recent reviews of energy
feedback solutions in research and practice indicate that no research has been conducted
on how CAs can be used to provide energy feedback [5, 7].
2.2 Conversational Agents
The idea of interacting with computers using natural language has been around for dec-
ades [8]. While the literature has used different terms to describe systems with conver-
sational user interfaces (e.g., CA, chatbot, or personal assistant), the underlying concept
is always that users “achieve some result by conversing with a machine in a dialogic
fashion, using natural language” [9]. In IS research, the most commonly used term is
“conversational agent” that refers to both text-based CAs, such as chatbots, and speech-
based CAs (e.g., Amazon’s Alexa) [19]. Both types of CAs build on the same technol-
ogy (i.e., natural language processing), but differ in their input/output modality (i.e.,
voice vs. text). CAs have their roots in the chatbot ELIZA [20] that was primarily de-
veloped to simulate human conversation based on pattern-matching algorithms. Since
then, the capabilities of CAs have improved enormously and many of them have been
implemented on websites and messenger platforms (e.g., for customer service). More-
over, they can be found on many mobile devices as personal assistants to support users
in finding information or accomplishing basic tasks (e.g., Apple’s Siri) [21].
CAs promise a more convenient and natural user interface than traditional graphical
user interfaces since they allow people to interact with computers using natural lan-
guage, just like engaging in a conversation with another person [8]. Particularly, less
IT-savvy users could benefit from this form of interaction because they do not need to
learn how to navigate through complex menus and understand detailed dashboards [8].
Moreover, CAs often display human-like characteristics (e.g., human-like appearance
or embodiment and communication style) to provide more natural and engaging inter-
actions [22] as well as to build relationships with users [23]. Therefore, CAs might also
serve as a natural way to provide energy feedback and promote sustainable energy use.
3 Design Science Research Project
This research project follows the DSR approach [12] to provide design principles (DPs)
for CAs for energy feedback promoting sustainable energy use in households. We argue
that this research approach is particularly suited to address our research goal because it
allows to iteratively design and evaluate our IT artifact in a rigorous fashion [12, 24].
Moreover, this approach enables us to involve experts and real users in the design and
evaluation phases to incrementally improve the functionality and relevance of our arti-
fact [12]. The project is conducted in collaboration with experts from an organization
in the energy industry. This organization is a medium-sized service provider that offers
a range of services, such as consulting, business process outsourcing, and product de-
velopment, for German energy providers and other companies in the energy industry.
The DSR project is based on the framework proposed by Kuechler and Vaishnavi
[24]. In the problem awareness phase, we reviewed extant literature on existing energy
feedback solutions to identify potential issues in their design. Based on the results of
this review, we proposed four DPs for CAs for energy feedback. These DPs were in-
formed by existing research on the design of energy feedback solutions and feedback
theory. Subsequently, we instantiated our DPs in an interactive prototype of a text-
based CA (i.e., a chatbot) developed with BotPreview, a platform for building previews
of chatbot interactions [25]. This prototype was then evaluated in an explorative focus
group session [13] with industry experts from the cooperating company. For the evalu-
ation, we selected the technical risk and efficacy strategy [26] because the implemen-
tation and evaluation of a CA for energy feedback in a real setting would be very costly.
The evaluation in a real household with real users would require significant investments
for setting up the necessary infrastructure (e.g., implementing a smart metering infra-
structure, integrating different data sources, and implementing algorithms for the cal-
culation of feedback) and recruiting participants. Therefore, we decided to first evaluate
the proposed DPs with a group of industry experts to get feedback and improve our
design before conducting a more complex evaluation. In a second design cycle, we will
refine our DPs based on the experts’ feedback and instantiate the DPs in a fully-func-
tional prototype. This prototype will be implemented using Microsoft’s Bot Framework
and evaluated with real users in several households that are equipped with smart meters.
4 Designing Conversational Agents for Energy Feedback
4.1 Problem Awareness
Feedback is considered a promising strategy for promoting sustainable energy use and
many energy feedback solutions have been developed in recent years [5]. Although
much research has been conducted on their design, there is still a need to better under-
stand and validate specific design features and interaction paradigms of these solutions
[27]. To inform our design, we conducted a literature review and identified several is-
sues in the design of existing energy feedback solutions, which we summarize below.
Many energy feedback solutions focus on visual feedback including numbers, text,
graphics, movement, animation, pictures, icons, colors, or lights [15]. However, these
solutions often overload consumers with too much information, dry numbers, and in-
tangible units [18]. In addition, they often lack natural language descriptions of key
information and a personal language that is easy to understand for consumers [7, 16].
Moreover, just providing information on energy use may not be sufficient for consum-
ers to draw conclusions for taking effective action (e.g., identifying energy guzzlers) or
changing energy use habits [16, 18]. Furthermore, many existing energy feedback so-
lutions either push out information to consumers (e.g., in-home displays positioned in
a visible place in the home) or require consumers to pull information (e.g., web portals
or mobile apps) [4, 17]. However, researchers argue that effective feedback solutions
should combine both push and pull approaches [17]. Furthermore, as energy is a low
involvement product [28] and energy feedback is usually optional for consumers [4],
there is a need to “design for the least motivated individuals” [17, p. 2]. However, many
energy feedback solutions cannot be easily integrated in consumers’ life or require a
complex system setup and training [6].
In conclusion, we argue that CAs represent a promising technology to address the
identified issues in the design of existing energy feedback solutions. While significant
progress has been made in the integration of real-time, appliance-level energy con-
sumption data (e.g., from smart meters) and the transformation of data into more com-
prehensible units (e.g., monetary savings) [e.g., 16], there is a lack of design knowledge
on CAs for energy feedback. Therefore, we believe that it is suitable to apply the DSR
approach to address this research gap.
4.2 Design Principles for Conversational Agents for Energy Feedback
In this section, we propose four DPs that describe how to design CAs for energy feed-
back. These DPs focus specifically on the CA and the way it should provide feedback
to consumers. In this paper, we do not further consider the underlying technical infra-
structure that is necessary to integrate different data sources (e.g., smart meters), nor
the algorithms that are necessary to, for example, calculate monetary savings or the best
time to start an appliance (e.g., washing machine). Research has made great strides in
developing the infrastructure and algorithms [e.g., 6, 16] that are required to implement
the technical basis of our DPs. However, since our main goal is to design a CA, we
argue that it is suitable to focus our DPs on how this technology can be used to provide
energy feedback. Next, we derive and formulate four DPs for CAs for energy feedback.
In general, CAs differ from other technologies in that they do not provide a typical
graphical user interface and rely on natural language as the main mode of interaction
[8, 9]. While text-based CAs, such as chatbots, are limited to a simple chat window,
voice-based CAs usually do not possess a graphical user interfaces at all. Consequently,
they are not able to show complex graphs, detailed statistics, or other visual elements
about current or past energy use. However, since consumers are able to chat with or
talk to them like having a conversation with another human being [8], they might pro-
vide a more natural user interface for energy feedback. Consumers should be able to
converse with a CA about their current and past energy use, ask specific questions about
their energy consumption choices, and receive personal feedback on their energy use.
For example, consumers could ask the CA about the current or past energy consumption
of a specific device or the best time to start their washing machine. Since this approach
might allow consumers to more quickly and effectively obtain answers to questions
about energy use (i.e., to pull information), the CA should provide comprehensible
feedback that enables them to draw conclusions on how to reduce or shift energy con-
sumption. Therefore, we propose:
DP1: Provide the CA with reactive energy feedback comprising comprehensible in-
formation in natural language in order to help consumers better understand their en-
ergy use and enable them to draw conclusions on how to use energy more sustainably.
However, reactive energy feedback provided by CAs should not be limited to only
providing comprehensible information (i.e., informative guidance [29]) but should also
include personalized suggestions and advice (i.e., suggestive guidance [29]). Research
argues that providing “highly personalized recommendations tailored to the sensed en-
ergy usage in the home” influences energy consumption behavior more effectively than
the graphical representation of consumption values or the provision of high-level writ-
ten or verbal messages [17, p. 6]. Therefore, CAs should provide reactive feedback that
includes suggestions and advice on how to reduce energy consumption (e.g., by identi-
fying energy guzzlers or “surprise” devices that they are unlikely to monitor [4]) and
shift times of consumption (e.g., rescheduling the washing process [3]). Moreover, CAs
should be able to support consumers in their decision to buy new energy-efficient de-
vices by performing complex cost/benefit analyses [17]. For example, consumers could
ask the CA whether buying a more energy-efficient refrigerator will reduce their energy
consumption and save them money in the long term by lowering their future electricity
bills. The CA could then support consumers in their purchase decision and even rec-
ommend suitable devices. Thus, we propose:
DP2: Provide the CA with reactive energy feedback comprising personalized sug-
gestions and concrete advice in order to enable consumers to act on it directly and
encourage sustainable energy use in the future.
In many domains, CAs show the promise of enhancing a user’s productivity by pro-
actively providing the information the user needs at the right time and at the right place
[30]. Similarly, research has demonstrated that energy feedback is much more effective
when delivered in the right context [31]. While DP1 and DP2 relate to reactive energy
feedback that requires consumers to pull information from the agent, CAs can also pro-
actively provide energy feedback to consumers (i.e., push information to the consumer).
For example, when a water heater is consuming excessive amounts of energy, the CA
should be able to promptly alert the consumer and suggest that there is a malfunction
so that s/he can take appropriate action. Additionally, the CA could send contact infor-
mation of a technician or apartment manager. Although solutions, such as mobile apps,
can also send proactive feedback using push notifications, we argue that CAs might be
more effective as their messages can serve as the starting point for a follow-up conver-
sation and thus, might foster deeper engagement with consumers.
However, while more frequent proactive feedback provides more opportunities to
engage consumers’ attention, there may also be an upper limit to the amount of time
that people are willing to spend on energy feedback [4]. Therefore, the CA should pro-
vide proactive feedback only in case of incidents that require the consumers’ attention
(e.g., device malfunction, anomalies in energy use, or significant money saving oppor-
tunities). Thus, we propose:
DP3: Provide the CA with proactive energy feedback comprising personalized sug-
gestions and concrete advice in order to enable consumers to quickly respond to
incidents that require special attention for a more sustainable energy use.
Finally, there is a rich body of knowledge that explores the design of human-like
characteristics for CAs. Following the “Computer are Social Actors” paradigm, many
studies have investigated how these social cues (e.g., human-like appearance or use of
natural language) enhance a CA’s trustworthiness and persuasiveness as well as make
the interaction more natural to users [32, 33]. Researchers argue that, to be effective in
persuasion, appropriate social cues should be embedded in the design of CAs [32]. So-
cial cues have also been found to increase the effectiveness of energy feedback [34].
For example, social feedback on the energy consumption of a washing machine pro-
vided by the social robot iCat was more persuasive than factual feedback provided by
an energy meter without any social cues [34]. Therefore, CAs for energy feedback
should also display social cues to make the human-CA interaction more natural and
their feedback feel more social. Thus, we propose:
DP4: Provide the CA with appropriate social cues in order to make the interaction
with them more natural and their energy feedback more social for consumers.
4.3 Artifact: Energy Feedback Agent (EFA)
Our proposed DPs were instantiated in an artifact called Energy Feedback Agent
(EFA). We decided to design EFA as a text-based CA (i.e., a chatbot) instead of a voice-
based CA because of the ubiquity of smartphones and the proliferation of instant mes-
saging applications [35]. More specifically, messaging has become a primary channel
for both personal and professional communication across all segments of the population
[9, 35]. Furthermore, since energy feedback may also contain sensitive information on
personal habits, consumers may not want others to hear the content of the feedback
[c.f., 36], which further supports the design as a text-based CA.
To instantiate our DPs, we selected two different scenarios based on examples in
existing literature [e.g., 17], which are explained in detail in section 5.1. Fig. 1 shows
the instantiation of DP1 and DP2 that illustrate how EFA provides reactive feedback
based on consumers’ questions. The left side of Fig. 2 depicts how EFA provides pro-
active feedback after an incident has been discovered that requires the consumer’s at-
tention (DP3). The right side of Fig. 2 shows the instantiation of DP4, that is, the social
cues that were implemented in EFA’s design. Based on insights from previous studies,
we selected several social cues to make the conversation more familiar to the user and
the provision of feedback more social, such as a human-like graphical representation
[37], small talk [23], and emojis [38].
Fig. 1. DP1 and DP2: reactive feedback (information and suggestions)
Fig. 2. DP3 (proactive feedback) and DP4 (social cues)
DP1 DP2
DP3 DP4
DP4
• Human-like
graphical
representation
• Addressing the
user by name
• Typing
indicators
• Emojis
• Small talk
5 Evaluation
5.1 Evaluation Methodology
To evaluate our proposed design, we conducted an exploratory focus group [39]. Ex-
ploratory focus groups have been used regularly in DSR to evaluate initial designs and
artifacts [e.g., 40, 41]. As shown in Table 1, the participants of the focus group session
were five employees of our partner organization and one employee of a major energy
provider. Upon arrival, the participants were asked to read and sign informed consent
forms, provide demographic information and answer three questions about their expe-
rience with smart metering technology as well as their use of CAs and messaging ap-
plications (using a Likert scale from 1=daily to 6=never). Our focus group consisted of
four males and two females, with an average age of 34 years and an average experience
with smart metering technology of 6 years. While most participants stated that they use
messaging applications daily (83%), their indicated use of CAs was only a few times a
month (50%) or even less (50%). Because of their broad industry experience and fa-
miliarity with smart metering technology, we argue that they can be regarded as indus-
try experts and represent an adequate sample for the evaluation as they are “familiar
with the application environment for which the artifact is designed so they can ade-
quately inform the refinement and evaluation of the artifact” [39, p. 127].
Table 1. Focus group participants
Participant
Affiliation
Business Unit
Expert 1 (EX1)
Service Provider
Product Management
Expert 2 (EX2)
Service Provider
Product Management
Expert 3 (EX3)
Service Provider
Business Development
Expert 4 (EX4)
Service Provider
Business Development
Expert 5 (EX5)
Service Provider
Sales
Expert 6 (EX6)
Major Energy Provider
Piloting & Operations
Two of the authors performed the focus group session, in which one researcher ac-
tively moderated the session, while the other one took notes throughout the session.
The focus group lasted a total of two hours and was structured as follows. First, the
moderator welcomed all participants and briefly explained the procedure of the focus
group session. After signing the informed consent forms, we started the audio record-
ing. Next, the participants were given an introduction on energy feedback and CAs.
Subsequently, we presented and explained our DPs for CAs for energy feedback and
demonstrated how they were instantiated in EFA. We used two scenarios to evaluate
the DPs one by one by showing how EFA would work in its intended environment.
The first scenario was used to evaluate EFA’s reactive feedback (i.e., DP1 and DP2).
In this scenario, participants should imagine that they are watching a news report on
the TV showing the consequences of climate change and therefore, wonder if they could
also do something to reduce their energy consumption. Realizing that they do not know
much about their current energy use, they open a messenger and start a conversation
with EFA. During the conversation, EFA answers several questions about current, past,
and average energy consumption (DP1). When EFA mentions that energy consumption
in the kitchen has been unusually high, they realize that a broken door (i.e., not closing
properly) of their refrigerator leads to a waste of energy. In this context, EFA also in-
dicates that the refrigerator is rather old and suggests buying a new, more energy-effi-
cient one to save energy and money in the long term (DP2). After stating their possible
interest, EFA calculates the time until the investment pays off and recommends two
suitable devices. Next, the same scenario was shown again, however this time, EFA
was designed to display social cues (DP4) as described in section 4.3. Apart from these
changes, however, the content of scenario was identical.
The second scenario was used to evaluate EFA’s proactive feedback (DP3). In this
scenario, EFA starts the conversation by alerting the consumers that their water heater
has consumed excessive amounts of energy over a long period of time, indicating a
technical problem with the device. Then, EFA suggests contacting a technician and
provides the phone number of a suitable technician to take care of the malfunction.
During the evaluation, we stopped the demonstration several times to explain how
the DPs were implemented. Therefore, participants could provide feedback on EFA and
the DPs at any time during the demonstration. After each demonstration, we asked
open-ended question about the artifact and the proposed design (e.g., “How did you
like the feedback provided by EFA?”). Depending on the course of the discussion, we
asked more specific questions about the proposed DPs and the interaction between EFA
and the consumer. After the session, we analyzed the participants’ feedback using our
notes and the audio recording. In the next section, we present the results of the analysis
and discuss the feedback of our focus group participants in detail.
5.2 Results and Discussion
In general, the industry experts who participated in our focus group session liked the
idea of using CAs to provide energy feedback to consumers in households. They
pointed out that EFA would be easier and more comfortable to use than many existing
feedback solutions because consumers would not need to install an additional app or
buy a new device. Moreover, since consumers frequently use instant messengers to
communicate with their friends and family members, EFA would be able to “pick up
consumers right where they always communicate” (EX6). The experts further argued
that the use of existing communication channels (e.g., Facebook Messenger or
WhatsApp) represents a low entry barrier for consumers, “especially for elderly people
or people with low IT affinity who might have difficulties installing an app” (EX3). In
addition, the experts stated that EFA would be of interest for energy providers looking
for products based on smart metering technology. Many consumers in Germany seem
to be skeptical of this new technology, but energy providers are legally required to im-
plement them on a large scale within the next years. Therefore, solutions like EFA could
help to reduce skepticism and facilitate the acceptance of smart meters in private house-
holds as “such feedback can help to provide an added value to the customer” (EX6).
Besides this general discussion, the experts also provided feedback on each DP. Con-
cerning DP1, one expert mentioned that “typical consumers have no relation to energy
consumption” (EX5) and their interest in finding out how to reduce or shift energy con-
sumption is rather low. Thus, when EFA provides clear answers to consumers’ ques-
tions immediately, it would “address the consumers at the right level” (EX6) and help
them to better understand the abstract and intangible concept of energy [15]. Another
expert liked that EFA “leaves the technical level” (EX1) of energy feedback by not only
using standard energy metrics, such as kilowatt-hours (kWh), but also metrics that are
well understood by consumers (e.g., € instead of kWh). Moreover, it was received pos-
itively that EFA provided consumers with a reference (e.g., by showing reference con-
sumption values, providing comparisons between devices, and explaining causal rela-
tionships). The experts argued that most consumers have difficulties to understand
whether a certain amount of energy indicates high or low consumption. Thus, they
found EFA’s ability to answer specific questions (e.g., “Is the energy consumption of
my fridge high?” or “Why is my consumption higher than last week?”) and provide
tailored feedback to be a great advantage. The experts believed that such feedback
would not only increase general energy literacy, but also motivate consumers to use
energy more sustainably (e.g., reduce or shift energy consumption) because they would
not have to invest the time and effort to look up information about their energy use
themselves. They suggested to go even further by identifying the consumer’s skill and
knowledge level and adapting EFA’s reactive feedback based on the consumer’s an-
swers to questions such as “Are you technically/commercially interested? Do you have
a technical background? Do you want the information in kWh or in €?” (EX6). For
example, inexperienced consumers would not be confronted with energy metrics at all,
while more knowledgeable consumers with a deeper interest in energy should also re-
ceive more complex feedback.
During the session, experts also stressed that energy feedback should not be limited
to the provision of pure information but should always include a possible explanation:
“With ‘25% more’ [i.e., energy consumption], it should be explained directly why this
could be the case, for example: ‘It could be the new device’” (EX5). Consequently, one
expert concluded that it is DP2 that makes the energy feedback effective. He argued
that when EFA provides personalized suggestions and advice on how to use energy
more sustainably, it would make it easier for consumers to respond to this feedback and
follow the suggestions rather than draw conclusions themselves. The experts also noted
that these suggestions should focus on small changes that can be implemented directly
rather than on overly complex or high-level advice. Furthermore, EFA’s ability to per-
form cost/benefit analyses using data from publicly available appliance databases [c.f.,
20] was regarded an important aspect of DP2 to help consumers understand the signif-
icant energy and cost savings potential of new energy-efficient devices. Again, the re-
duction of effort for consumers was positively evaluated (e.g., consumers would not
need to search for suitable devices themselves).
In general, the experts also liked the fact that EFA “pushes” feedback proactively to
consumers (DP3). They argued that EFA should not remain passive because, after some
time, consumers naturally begin to disengage with an energy feedback solution [42].
However, one expert argued that proactive feedback should always “include concrete
suggestions and advice so that [one] can rule out possible causes” (EX2). This point
was also addressed by another expert who criticized that a consumer “[needs] more
information in addition to the message to act accordingly” (EX1). Furthermore, they
would not want to receive daily reports on their energy use from EFA, but rather spe-
cific messages as a reaction to an important event or incident. However, one expert
suggested that EFA should follow up on proactive feedback if consumers do not re-
spond (e.g., “for less important events, a continuous reminder should come up” (EX1)).
In case of emergencies (e.g., when an oven malfunctions), EFA could even make an
automatic phone call to a dedicated emergency number.
The industry experts also believed that appropriate social cues displayed by EFA
(DP4) would help to increase consumer engagement and make the energy feedback
appear more natural. For example, one expert stated that, by displaying social cues,
“EFA tends to come across as a friend; the flow is more natural and maintains com-
munication. In the second example [i.e., with social cues / DP4], [he] would have asked
more questions than in the first example [i.e., without DP4]” (EX1). Moreover, incor-
porating social cues, such as emojis, also helps to make the conversation appear more
familiar to consumers and thus, might increase feedback effectiveness. However, one
expert cautioned that these social cues should be designed carefully to not distract from
EFA’s main purpose to provide energy feedback: “The bot should be less cheeky and a
little more formal because it’s about money” (EX3). Table 2 summarizes the key find-
ings of our focus group discussion with industry experts.
Table 2. Summary of key findings of the focus group discussion
DP
Key Findings
DP1
• Provides easy access to information on energy use in natural language
• Facilitates consumers’ understanding by quickly giving comprehensible answers
• Could be individually adapted to the consumer’s skill/knowledge level and preferences
DP2
• Helps consumers to directly respond to feedback by pointing out specific measures
• Reduces effort for consumers to find out how to use energy more sustainably
DP3
• Facilitates re-engagement or continued interaction with EFA
• Needs to include further information about the potential causes of an incident
DP4
• Encourages consumers to interact with EFA (e.g., ask more questions)
• Needs to be designed carefully to not distract from EFA’s main purpose
The focus group discussion also brought up some interesting aspects about the mo-
dality (i.e., text vs. voice) used by CAs for energy feedback. One expert suggested that
consumers should be able to communicate with EFA using text messages (e.g., on the
phone when they are not at home) and using voice input (e.g., if they own a device like
Amazon Alexa). Moreover, they argued that, in some cases, using a voice-based EFA
would further reduce the effort for seeking energy feedback since consumers do not
need to enter a text message. One expert mentioned that the modality (text vs. voice)
could also be automatically selected based on the consumer’s current location. Moreo-
ver, EFA’s functionality could be extended to be able to turn devices on and off, similar
to existing smart home solutions.
In conclusion, the industry experts believed that with CAs, such as EFA, energy
feedback could take an important step into consumers’ daily life and help them to use
energy more sustainably. Moreover, they argued that such a solution would provide
energy providers with the opportunity to offer their customers a benefit from smart
metering technology, which further indicates the relevance of our proposed design for
a real-world context. According to the experts, technological advances within the next
years will make it possible to easily extract and integrate the data that is required to
provide the basis for the implementation of our design. However, they also noted that
EFA needs to possess advanced natural language processing capabilities to provide ac-
curate feedback and ultimately, to ensure adoption and continued use by consumers.
6 Conclusion
This paper presents the findings of our DSR project on how to design CAs for energy
feedback to promote sustainable use of energy in households. We identified several
issues in the design of existing energy feedback solutions and proposed four DPs to
address these issues by designing a CA for energy feedback. We instantiated our DPs
in a text-based CA called EFA and evaluated it in an exploratory focus group session
with industry experts. Overall, the results of our evaluation indicate that CAs represent
a promising technology for energy feedback and designing these CAs based on our DPs
could enable consumers to use energy more sustainably. We therefore contribute with
valuable design knowledge that extends previous research on energy feedback solutions
and serves as a starting point for future research on designing CAs for energy feedback.
Although our research follows established guidelines for conducting DSR [12, 24],
there are some limitations that need to be discussed. First, we instantiated our DPs in a
text-based CA (i.e., chatbot). However, as also illustrated in the feedback by industry
experts, voice-based CAs seem to be a promising medium for energy feedback as well,
possibly even in combination with a text-based CA. Therefore, future research could
instantiate and evaluate our DPs in a voice-based CA such as Amazon’s Alexa. More-
over, the evaluation was conducted with industry experts who might be biased because
of their familiarity with energy feedback solutions. Thus, another focus group session
with real, non-expert users could provide an important complementary perspective on
our DPs. Finally, we used an interactive prototype without real data or algorithms to
demonstrate EFA’s capabilities in two scenarios. Although we argue that this approach
is appropriate for a first evaluation of EFA, further research implementing a full infra-
structure is needed. Therefore, we plan to implement a fully functional prototype in
several households and perform a field-based evaluation study in our future research.
References
1. International Energy Agency (2017) World Energy Investment 2017.
https://www.iea.org/publications/wei2017/. Accessed 28 Jan 2018
2. European Commission (2017) 2050 Low-Carbon Economy Roadmap.
https://ec.europa.eu/clima/policies/strategies/2050_en. Accessed 15 Jan 2018
3. Kobus CBA, Mugge R, Schoormans JPL (2013) Washing when the sun is shining! How
users interact with a household energy management system. Ergonomics 56:451–462
4. Karlin B, Zinger JF, Ford R (2015) The Effects of Feedback on Energy Conservation: A
Meta-Analysis. Psychol Bull 141:1205–1227
5. Karlin B, Ford R, Squiers C (2014) Energy feedback technology: A review and taxonomy
of products and platforms. Energy Effic 7:377–399
6. Weiss M, Helfenstein A, Mattern F, Staake T (2012) Leveraging Smart Meter Data to
Recognize Home Appliances. In: 2012 IEEE International Conference on Pervasive
Computing and Communications. IEEE, pp 190–197
7. Pullinger M, Lovell H, Webb J (2014) Influencing household energy practices: a critical
review of UK smart metering standards and commercial feedback devices. Technol Anal
Strateg Manag 26:1144–1162
8. McTear M, Callejas Z, Griol D (2016) The Conversational Interface: Talking to Smart
Devices. Springer
9. Dale R (2016) The return of the chatbots. Nat Lang Eng 22:811–817
10. Gartner (2017) Top Trends in the Gartner Hype Cycle for Emerging Technologies.
https://www.gartner.com/smarterwithgartner/top-trends-in-the-gartner-hype-cycle-for-
emerging-technologies-2017/. Accessed 20 Dec 2017
11. Bourgeois J, Linden J Van Der, Kortuem G, van der Linden J, Kortuem G, Price BA,
Rimmer C (2014) Conversations with my washing machine: an in-the-wild study of demand-
shifting with self-generated energy. In: Proceedings of the 2014 ACM International Joint
Conference on Pervasive and Ubiquitous Computing. pp 459–470
12. Hevner AR, March ST, Park J, Ram S (2004) Design Science in Information Systems
Research. MIS Q 28:75–105
13. Tremblay MC, Hevner AR, Berndt DJ (2010) Focus Groups for Artifact Refinement and
Evaluation in Design Research. Commun Assoc Inf Syst 26:599–618
14. Kluger AN, DeNisi A (1996) The Effects of Feedback Interventions on Performance: A
Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory.
Psychol Bull 119:254–284
15. Sanguinetti A, Dombrovski K, Sikand S (2018) Information, timing, and display: A design-
behavior framework for improving the effectiveness of eco-feedback. Energy Res Soc Sci
39:55–68
16. Dalén A, Krämer J (2017) Towards a User-Centered Feedback Design for Smart Meter
Interfaces to Support Efficient Energy-Use Choices. Bus Inf Syst Eng 59:361–373
17. Froehlich J (2009) Promoting Energy Efficient Behaviors in the Home through Feedback:
The Role of Human-Computer Interaction. In: Proc. HCIC Workshop
18. Weiss M, Staake T, Mattern F, Fleisch E (2012) PowerPedia - Changing Energy Usage with
the Help of a Smartphone Application. Pers Ubiquitous Comput 16:655–664
19. Gnewuch U, Morana S, Maedche A (2017) Towards Designing Cooperative and Social
Conversational Agents for Customer Service. In: Proceedings of the 38th International
Conference on Information Systems (ICIS). Seoul, South Korea
20. Weizenbaum J (1966) ELIZA - A Computer Program for the Study of Natural Language
Communication between Man and Machine. Commun ACM 9:36–45
21. Maedche A, Morana S, Schacht S, Werth D, Krumeich J (2016) Advanced User Assistance
Systems. Bus Inf Syst Eng 58:367–370
22. Beale R, Creed C (2009) Affective interaction: How emotional agents affect users. Int J Hum
Comput Stud 67:755–776
23. Bickmore T, Cassell J (2001) Relational Agents: A Model and Implementation of Building
User Trust. In: Proceedings of the 2001 SIGCHI Conference on Human Factors in
Computing Systems
24. Kuechler B, Vaishnavi V (2008) Theory Development in Design Science Research:
Anatomy of a Research Project. Eur J Inf Syst 17:489–504
25. Gall M (2018) BotPreview.com. https://botpreview.com/. Accessed 28 Jan 2018
26. Venable J, Pries-Heje J, Baskerville R (2016) FEDS: a Framework for Evaluation in Design
Science Research. Eur J Inf Syst 25:77–89
27. Miller W, Senadeera M (2017) Social transition from energy consumers to prosumers:
Rethinking the purpose and functionality of eco-feedback technologies. Sustain Cities Soc
35:615–625
28. Watson A, Viney H, Schomaker P (2002) Consumer attitudes to utility products: a consumer
behaviour perspective. Mark Intell Plan 20:394–404
29. Morana S, Schacht S, Scherp A, Maedche A (2017) A Review of the Nature and Effects of
Guidance Design Features. Decis Support Syst 97:31–42
30. Sarikaya R (2017) The Technology Behind Personal Digital Assistants: An Overview of the
System Architecture and Key Components. IEEE Signal Process Mag 34:67–81
31. Tiefenbeck V, Goette L, Degen K, Tasic V, Fleisch E, Lalive R, Staake T (2016)
Overcoming Salience Bias: How Real-Time Feedback Fosters Resource Conservation.
Manage Sci 1–19
32. Fogg BJ (2002) Computers as Persuasive Social Actors. In: Persuasive Technology: Using
Computers to Change What We Think and Do. Morgan Kaufmann Publishers, San
Francisco, CA, USA, pp 89–120
33. Nass C, Steuer J, Tauber ER (1994) Computers are social actors. In: Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems. Boston, MA, USA, pp 72–
78
34. Ham J, Midden CJH (2014) A Persuasive Robot to Stimulate Energy Conservation: The
Influence of Positive and Negative Social Feedback and Task Similarity on Energy-
Consumption Behavior. Int J Soc Robot 6:163–171
35. Statista Number of mobile phone messaging app users worldwide from 2016 to 2021. In:
2018. https://www.statista.com/statistics/483255/number-of-mobile-messaging-users-
worldwide/. Accessed 28 Jan 2018
36. Easwara Moorthy A, Vu KPL (2015) Privacy Concerns for Use of Voice Activated Personal
Assistant in the Public Space. Int J Hum Comput Interact 31:307–335
37. Appel J, von der Pütten A, Krämer NC, Gratch J (2012) Does Humanity Matter? Analyzing
the Importance of Social Cues and Perceived Agency of a Computer System for the
Emergence of Social Reactions during Human-Computer Interaction. Adv Human-
Computer Interact 2012:1–10
38. Klopfenstein LC, Delpriori S, Malatini S, Bogliolo A (2017) The Rise of Bots: A Survey of
Conversational Interfaces, Patterns, and Paradigms. In: Proceedings of the 2017 Conference
on Designing Interactive Systems. pp 555–565
39. Tremblay MC, Hevner AR, Berndt DJ (2010) The use of focus groups in design science
research. In: Hevner A, Chatterjee S (eds) Integrated Series in Information Systems. Design
Research in Information Systems. Springer US, Boston, MA, USA, pp 121–143
40. Morana S, Schacht S, Scherp A, Maedche A (2014) Designing a Process Guidance System
to Support User’s Business Process Compliance. ICIS 2014 Proc 1–19
41. Zheng G, Vaishnavi VK (2011) A multidimensional perceptual map approach to project
prioritization and selection. AIS Trans Human-Computer Interact 3:82–103
42. Snow S, Buys L, Roe P, Brereton M (2013) Curiosity to cupboard: self reported
disengagement with energy use feedback over time. In: Proceedings of the 25th Australian
Computer-Human Interaction Conference. pp 245–254
