ArticlePDF Available

A Strategy Framework to Boost Conversational AI Performance

Authors:

Abstract

The development of conversational AI applications is often more tactical than strategic; more technology-focused than focused on the end user and the problem to be solved. This paper provides a strategy framework to structure the effective development of conversational AI in research and practice.
www.marketing-review.ch
Marketingzeitschrift für Theorie & Praxis
4 | 2021 SCHWERPUNKT A Strategy Framework for the Design of Conversational AI Digitale Empathie
von Conversational Interfaces Power Attributions of Conversational Agents Based on Vocal Features
The Emerging Business of Online Lead Generation SPEKTRUM Smart Services and Customer Expectations
in the Automotive Sector A Motivational Framework for Gamication Integrierte Unternehmensführung
als Perspektive bei Ungewissheit
Conversational
Commerce
MRSG_4-2021_Titel_m.indd 1MRSG_4-2021_Titel_m.indd 1 16.06.21 11:3016.06.21 11:30
Schwerpunkt Überblick zur Gestaltung von Conversational AI
A Strategy Framework
to Boost Conversational
AI Performance
The development of conversational AI applications is often more tactical than strategic;
more technology-focused than focused on the end user and the problem to be solved.
This paper provides a strategy framework to structure the effective development of
conversational AI in research and practice.
Prof. Dr. Christian Hildebrand, Sophie Hundertmark
10 Marketing Review St. Gallen 4 | 2021
010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 10010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 10 17.06.21 08:3517.06.21 08:35
Schwerpunkt Überblick zur Gestaltung von Conversational AI
Conversational A I is fundamentally changing how
consumers interact with firms. With conversa-
tional AI ranging from text-based chatbots auto-
mating simple customer service tasks to digital voice as-
sistants providing active recommendations during a
shopping trip, conversational AI provides a scalable tech-
nology that is poised to cr eate hy per-per sonalized and
contextualized customer experiences that were unthink-
able with traditional technologies (Hildebrand & Bergner,
2020; Zum stein & Hu ndertmark, 2018). As mess aging
platforms have become the d igital de-facto standard for
consumers to interact with their friends and families
(Economist, 2016), consumers shy away from traditional
consumer–fir m communicat ion modes and dema nd faster
and more personalized ways of interaction. Recent indus-
try reports highl ight the strong cost-saving potential of
conversational AI during the COVID-19 pandemic while
academic research revealed that digital voice assistants
can be engineered to be as effective as the top 20% of hu-
man sales representatives (Luo et al., 2019; Maruti
Tec hl ab s, 2017 ). Ho wev er, co nsu mer s of ten experie nce
frustrat ion (e.g., due to a lack of understanding of what the
consumer is saying), express privacy concerns, or experi-
ence heightened uncertainty when interacting with con-
versational AI applications (Hildebrand et al., 2020; Shu-
manov & Johnson, 2021; Thomaz et al., 2020). As
illustrated in the exa mples in panels A and B of figure 1,
despite all chatbots using non-human avatars, both Revo-
lut and Helvet ia provide the requested inform at ion in
short, clear sentences, do not forward the user to a sepa-
rate website, directly react and respond to the user i nput
as a human would and with the appropriate tonality (even
using emojis as a reaction to the posit ive user feedback in
the Revolut example).
As these examples show, the design logic in current
industry applications varies tremendously and signifi-
cantly impacts the user experience. To address this issue,
the current paper provides a strategy framework for de-
veloping more effective conversational AI applications.
Specifical ly, we integrate pr ior work on t he effec tive
design of information systems and user-centric customer
journeys to deliver more effective conversational AI ex-
periences. This model offers a st rategic perspective on
the effective design of conversational AI, in contrast to
the dominant technology focus in recent work on conver-
sational AI (see Rapp et al., 2021), and mitigates common
issues along the design, implementation, and deployment
phases.
Prof. Dr. Christian Hildebrand
Full Professor of Marketing Analytics and
Director of the Institute of Behavioral
Science & Technology and the TechX Lab
at the University of St. Gallen, Switzerland
christian.hildebrand@unisg.ch
Sophie Hundertmark
Speaker and consultant on digital
transformation and chatbots, and external
doctoral candidate at the Institute of
Financial Services Zug (IFZ) at the Lucerne
University of Applied Sciences and Arts,
Lucerne, Switzerland
sophie@hundertmark.ch
www.hundertmark.ch
The Six-Stage Model of Effective
Conversational AI
The model captures six distinct stages of effective conver-
sational AI design summarized in this paper. It integrates
prior work on the strategic design of information systems
(Baptista et al., 2021; Gable, 2020) and the effective design
of user-centric customer journeys in marketing (Lemon &
Verhoef, 2016).
We fi rst iso late the pr oblem to be solv ed, followe d by de-
fining the expected target user. Next, we discuss the critical
role of integrating different digital channels and the specific
design decisions in the implementation and deployment phase.
Finally, we outline the onboarding of the internal stakeholders
and the selection of the appropriate technology stack. While
this paper focuses on optimizing consumer–firm interactions,
the model also applies to any other non-consumer sett ing
(su ch as devel oping a conve rsa tio nal AI fo r i nterna l pr oce ss-
es). Figure 2 provides a visual illustration of the distinct stag-
es and key questions in each development stage.
tae   A ene te er rolem
The first stage requires the definition of the specific task
that should be automated by the conversational AI. Al-
though this first stage sounds obvious, industry reports sug-
gest a strong focus on the technology infrastructure, inter-
nal processes, and governance topics at the out set of a
11
Marketing Review St. Gallen 4 | 2021
010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 11010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 11 17.06.21 08:3517.06.21 08:35
Schwerpunkt Überblick zur Gestaltung von Conversational AI
project instead of focusing on the type of problem that should
be solved for customers (Nguyen, 2017). We recom mend
analyzing the entire customer journey and critically evaluat-
ing in which stage conversational AI applications contribute
to the existing objectives of the business. One dichotomy that
helps with prioritization is to critically evaluate whether the
trajectory of the business is focused on growth or cost-cut-
ting. For example, if the COVID-19 pandemic requires a
strong focus on costs, gaining management support for a pilot
focusing on optimizing product information is less likely to
receive approval compared to automating call center services
through conversational AI to reduce service operation costs.
Our recommendation is to first outline a list of specific
tasks with high potential for automation, then prioritize the
selected use cases or tasks from a feasibility and business
value standpoint. Suppose a company came up with three
potential pilot project s in which options to solve the user
problem ranged from (1) automating parts of the customer
service process through a chatbot, (2) providing automated
decision support by offering additional product information
on a website through a chatbot, or (3) actively providing rec-
ommendations during the sales process through a chatbot.
While the first project is arguably highly feasible, the ex-
pected business value is comparatively low. In contrast, the
last project has a comparatively high business value but
lower feasibility as recommendations have to be highly indi-
vidualized and contextualized. Finally, the clear articulation
of the problem to be solved should also be closely t ied to
defining how the firm will track progress toward that objec-
tive (such as assessing customer acquisition costs before and
after the pilot or the duration of customer ser vice calls, de-
pending on the key objective of the conversational AI). In
summary, the first stage requires an exclusive focus on the
problem that should be addressed from the user perspective.
tae    ene te aret er
The second stage is designed to define the envisioned target
user. We recommend the use of methods already employed in
prior work on customer journey mapping by defining the “ar-
chetypical user persona” (Lemon & Verhoef, 2016). Thus, the
company should evaluate the predominant user of the conver-
sational AI ser vice for the defined problem (stage 1). The ex-
pected persona/s should be as specific as possible and cover
both objective characteristics such as demographics (age, gen-
der, marital status) and subjective characteristics such as the
expected emotional state or important values of the user. An-
ticipating the emotional state of a user is critical as a negative
experience of an already frustrated user can lead to a negative
downward spiral and a negative evaluation of the firm. For
Source: Own i llustrati on.
Fig. 1: Selected Examples of Chatbots
Revolut Financial
Servicebot
Asiana Airlines
Travelbot
Helvetia Switzerland
Insurancebot
Patient Infobot
„116117“
A B
12 Marketing Review St. Gallen 4 | 2021
010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 12010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 12 17.06.21 08:3517.06.21 08:35
Schwerpunkt Überblick zur Gestaltung von Conversational AI
next. This could imply the seamless integration of human
sales representatives taking over. Failed intent matching and
ineffective channel integration are major sources of frustra-
tion for conversational AI users due to the reduced sense of
goal attainment (Leung & Chan, 2020). Thus, objectives of
the third stage are to decide which channel to focus on and to
consider potential interactions across all channels and touch-
points. Companies are well advised to carefully assess failed
intent matchings not only before or during a pilot phase but
also at regular time intervals after the conversational AI ap-
plication has gone live. The majority of conversat ional AI
interfaces provides identifiers that explicitly flag failed inter-
actions or intents in the backend, which can (and should) be
carefully and systematically analyzed.
example, recent research by Hadi (2019) revealed that custom-
ers who were in a state of anger or frustration before entering
the interaction with a customer service chatbot were even more
frustrated after the interaction and evaluated the firm more
negatively compared to a group of cont rol customers. Thus,
firms are advised to carefully define the expected target user
and to incorporate the anticipated emotional state of the user.
tae    ene te annel nteration
Acro ocoint
The third stage is designed to assess and decide how the con-
versational AI application should be deployed. This can range
from integration on an existing website or within an existing
third-party platform (WhatsApp, WeChat, Facebook, etc.) to
deployment as a stand-alone App. This stage requires a deci-
sion regarding the core channel of operation and the question
how channel switching should be handled across touchpoints.
For example, a firm might decide to implement a conversa-
tional AI as part of their sales automation processes on the
website. Thus, integration and deployment require a decision
with respect to where on the website the conversational AI
should be made available and what should happen in the case
of failed intent matchings (i.e., when the conversational AI
does not know how to answer or handle a prospect request).
A failed intent matching should move a prospect or existing
customer seamlessly from one channel of interaction to the
Source: Own i llustrati on.
Fig. 2: Six-Stage Model of Conversational AI Design
What are the technical
requirements, including
data protection
regulations? Which bot

organization and the
use case best (native
internal development
vs. third-part y cloud
technology)?
WITH WHAT:
ene te
ecnolo
Stack
Stage 6

problem should
the chatbot solve?
Which goals should
be achieved with
the bot and how
will success be
measured?
WHAT:
ene te
er rolem
Stage 1
Where should the
chatbot be inte-
grated? Through
which channel will
our target group
want to use the
chatbot (integration
on website vs. third
party platform vs .
app etc.)?
WHERE:
ene te
annel
nteration
Stage 3
Who will use the
bot? In which
situations will it
be used? In which
emotional state
will the user access
the bot?
WHO:
ene te
aret er
Stage 2
What tonality does
the chatbot use?
What is its name?
If the bot were a
human, what
personality would
it have?
HOW:
ein te
Aearance
onalit an
eronalit
Stage 4
Who will be par t
of the extended
project team?
Which internal stake-
holders need to be
met and convinced
at which point in
time? What is the
project timeline?
WITH WHOM:
entif te
tene eam
 e take
oler
Stage 5
Management Summary
Conversational AI is fundamentally changing how consumers


user experience. To address this issue, the current paper provides
a strategy framework for developing more effective conversatio-
nal AI applications. This model offers a strategic perspective on
the effective design of conversational AI and mitigates common
issues along the design, implementation, and deployment phases.
13Marketing Review St. Gallen 4 | 2021
010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 13010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 13 17.06.21 08:3517.06.21 08:35
Schwerpunkt Überblick zur Gestaltung von Conversational AI
tae    ein te Aearance
onalit an eronalit of te A
The fourth stage encompasses all design-related decisions
and involves the visual appearance and the conversational
design (the structural and semantic properties of how the
conversation between a human user and the AI is governed).
For e xa mple, t he visual design in the contex t of chatbots
involves the company or brand-congruent design of the digital
avatar, the name of the avatar, the coloring and font type. The
selection of the avatar and the appropriate naming are critical
design choices (Miao et al., 2021) as they directly reflect the
brand. The conversational design captures the structural and
semantic properties of the conversation. The structural di-
mensions entail, for example, the frequency or extent of turn-
taking (i.e., whether the conversational AI actively promotes
a back-and-forth communication as in human-to-human com-
munication). Recent research has demonstrated that a greater
extent of turn-taking enhances trust and a more positive
evaluation of the brand (Hildebrand & Bergner, 2020). The
semantic dimension captures the tonality of the conversa-
tional AI and can range from a more formal to an informal
tone of communication (using, for instance, more affect-rich
language or emojis). The combination of the visual and con-
versational design ultimately defines the type of personality
users will attr ibute to the conversational AI. For exa mple,
more affect-rich language can be used intentionally to engi-
neer a more extraver ted “personality” of the conversational
AI. Such personality attributions can even be evoked by more
subtle cues such as leaving longer pauses to indicate a greater
sense of reflectivity on the part of the AI or increasing the
variability of vocal frequency to indicate excitement (Hilde-
brand et al., 2020). In short, human users tend to attribute
distinct personalities to the conversational AI, and the sys-
tematic visual and conversational design are key factors in
engineering the envisioned personality profiles of the firm or
brand (Nass & Moon, 2000; Nass et al., 1994).
tae    M ene te tene
roect eam  takeoler
The fifth stage focuses on defining the extended project team
and on actively onboarding internal stakeholders. The initia-
tive typically resides in one functional organization (for ex-
ample, the HR department when it comes to the use of a chat-
bot for internal trainings of employees or the marketing
department regarding the use of a chatbot as par t of their
digital marketing act ivities to generate and convert leads).
This stage is essential for winning over the entire organiza-
tion and extended project team (all internal stakeholders that
will either directly or indirectly be involved in the project,
such as the internal IT department). The fact that conversa-
tional AI applications are based on relatively recent techno-
logical developments can lead to inter nal resistance that needs
to be actively managed. This stage is critical for avoiding
false expectations while identifying strong internal promoters
of t he pilot project. It is vital to ant icipate and respond to
potential internal resistance. For example, it is recommend-
able to actively communicate how the success of the pilot is
measured (for example, in the context of a sales automation
project; this may involve the amount of traffic to key landing
Lessons Learned
1. Start with the problem and end user in mind,
not the technology.
2. Consciously design all user-relevant touch-
points and the integration of human emplo-
yees and the conversational AI .
3. Design the tonality and personality to maximi-
ze a coherent brand experience for the user.
4. Actively integrate internal and external
 
timelines, and measures of success.
Main Propositions
1. Conversational AI allows to build and nurture

2.
reduce the risk of project failure or low
acceptance rates among users, we propose a
structured process to develop conversational
AI applications in research and practice.
3.
case, the target user, the digital touchpoints,
the visual and conversational design characte-

and decide on the technology stack.
14 Marketing Review St. Gallen 4 | 2021
010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 14010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 14 17.06.21 08:3517.06.21 08:35
Schwerpunkt Überblick zur Gestaltung von Conversational AI
pages, actual conversion rates on those landing pages, or the
number of pages visited before the key landing page).
tae    A ene te ecnolo tack
The sixth stage focuses on the selection of the most appropriate
technology stack in view of all the preceding stages. The deci-
sion on the technology stack is positioned at the last stage of
the model for two major reasons. First, the technology stack
should be selected according to the specific use case, independ-
ent of the internal processes and already existing infrastruc-
ture. This sequence is designed to avoid narrowing down the
options and distracting from the focal problem of the end user
as opposed to the technology. Second, focusing on the technol-
ogy after the requirements regard ing the conversational AI
helps to critically evaluate whether the envisioned use case
requires a complex natura l language processing engine or
whether a simple rule-based conversational agent is sufficient.
We acknowledge, however, that t hi s stage also has to
incorporate the organizational requirements such as previ-
ously used technologies, existing data protection regulations,
and other project resources. The presented model may be
seen as an ideal archetype, and the actual implementation of
a project may require several iterations, going back from the
technology to earlier stages. As the number of conversation-
al AI providers cont inues to grow (ranging from Amazon,
Google, Microsoft, and IBM to smaller, more specialized
solution providers), we recommend to regularly monitor the
development on the technology side. As with any corporate
information systems project, large companies usually offer
highly scalable but more standardized solutions whereas
smaller providers are often more actively involved and offer
a better, customized fit.
Future Directions in Research and
Industry Practice
We see four frui tful dire ction s for fut ure work in res ear ch
and practice. First, future work could further explore the in-
teractions between the individual stages highlighted in this
paper. Specifically, future work may further assess the inter-
play between the users’ psychological predispositions and
their specific design preferences of the conver sational AI.
For example, users with higher levels of extraversion might
prefer a conversational agent with a larger number of affect-
rich, emotiona l cues (from emojis in written language with
text-based chatbots to greater vocal variability in digital
voice assistants) (Hildebrand et al., 2020).
Second, future work may further explore the role of adap-
tive user interfaces (de Bellis et al., 2019) and the importance
of more contextualized interactions. For example, a user with
higher levels of extraversion might prefer a more emotional
voice assistant when searchi ng for a new fashion item or
screening a new movie on Netflix. However, the same user
may prefer a less emotional voice assistant when searching
for a mortgage or optimizing their financial portfolio. Thus,
future work could further explore the importance of dynami-
cally adapting the conversational AI to the context or task.
Third, future work could further explore the dynamics of
the strategic framework provided in this research over time.
Specifically, the type of problems that a bot is expected to han-
dle is a ‘moving target’ and may change in both scope and
complexity over t ime (stage 1). Due to the implementation
across multiple channels the end user base is often becoming
more diverse, and consumer expectations are likely to increase
as well (stage 2). Also, the digital channel landscape is con-
tinuously evolving, raising the question of effective channel
integration across touchpoints (stage 3). Finally, consumer ex-
pectations and legal frameworks may require developers in the
future to use non-humanized avatars to avoid the impression
that the agent is a human as opposed to a machine (stage 4).
Four th, t he framework does not specify a n explic it “moni-
toring stage” but in practice this part is of critical importance.
Specifically, companies must not forget to monitor the conver-
sational assistant and improve it on the basis of important per-
formance met rics such as bounce rate, user feedback, and the
questions received (particularly those that lead to a dysfunc-
tional outcome or dead end for the user). Effective conversa-
tional AI is not a fixed product or service but in constant flux
with changing user demands. Usually, a chatbot does not cover
all of the users' relevant questions immediately after the launch.
These must be supplemented with further iterations. If the com-
pany has decided on a very distinctive chatbot personality, this
must also be critically scrutinized after a certain operating time
and possibly adapted depending on user reactions.
Finally, the majority of emerging work on conversation-
al AI in marketing and related areas has focused on studying
Effective conversational AI
is not a fixed product or
service but in constant flux
with changing user demands.
15Marketing Review St. Gallen 4 | 2021
010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 15010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 15 17.06.21 08:3517.06.21 08:35
Schwerpunkt Überblick zur Gestaltung von Conversational AI
relatively narrowly defined contexts such as specific cus-
tomer serv ice ta sks (Hadi, 2019; H ildebrand & Berg ner,
2020; Thomaz et al., 2020). There are ample opportunities for
more conceptua l work based on the current paper to help
synthesize the emerging effects into broader organizing
frameworks a nd design pri nciples. This work would help
both to integrate and synthesize the quickly emerging em-
pirical work into larger conceptual frameworks and to high-
light potential blind spots in empirical research and give
directions for future research.
Conclusion
Conversational AI is fundamentally changing how consum-
ers search, shop, and express their preferences. Despite the
oppor tunity to del iver cost-effective and hig hly sca lable
user experiences, emerging research and industry findings
are mixed and range from greater user satisfaction to height-
ened frustration and reluctance to engage in fut ure use of
conversational AI due to privacy concerns and poor intent
matching. This paper provides a strategic framework for de-
veloping more effective conversat ional AI based on prior
work on the effective design of in formation systems. We
provide a structured process that is designed to mitigate
common issues in the design, development, and deployment
phase by first clarifying the intended use case, then defining
the target user, the integrated digital channels, the visual
and conversational design characteristics of the conversa-
tional agent, the onboarding phase with internal stakehold-
ers, and finally the envisioned technology stack. We hope
that this research inspires future work in the quickly devel-
oping field of conversationa l AI at the intersection of mar-
keting, infor mation systems, and the user-centric design of
conversational AI applications.
Literature
atita   ilon A    Gallier   
  ntanti ation econceta liin
te role of tecnolo a a carr ier of
orani ational tratei in  ornal of
nformation ecnolo
e elli   ileran   to   er rmann
A  cmitt    eronalii n te
ctomiation ee rience a matcin
teor  of ma c tomiatio n interface an
cltral i nformati on rocein orna l of
Marketin eearc     
conomi t    Aril ot  te
net fro ntier e co nomit
ttwwweconomitcomnew
ineannancemarket
amatrinnowonetetae
erviceorcatotlookoie
Gale G  G  e at an ftre of
e or nal of trateic nfor mation te m
A convera tion wit o Gallie r ornal of
trateic nformation tem   
ttoiori 
ai    en  maniin ctome r
ervi ce catot m it ack re M
Marketin ntelli ence eview   
ttoiornimmir
ile ran    erner A 
onverational roo avior a  rroate  of
trt onoarin eerience rm ercetion
an con mer na ncial eci ion makin
orna l of te Acae m of Marketi n cience
ttoior
ile ran   ftmio F et  F
amton     offman      ovak  
 oice an alti c in ine reearc
oncetal fona tion a cotic fe atre
etrac tion an  alication  ornal of
ine eear c    
ttoiorre 
emon    eroef    ner
tanin ctomer eerience an te ctomer
orne ornal of Marketin  
en     an     et ail
catot  e callene an  oor tnitie of
convera tional commerce orna l of iita l
an ocia l Meia Mar ketin    
o   on   Fan     
Macine ver man t e imact
of A catot i clor e on ctom er
rcae Mar ketin cie nce 
ttoiormkc
Marti ecla   an catot el
rece c tomer ervice co t  
atot Ma aine  ttcatot 
maainecomowwitteelof
catotctomerervicecotcol
erecetoa
Miao F olenkova   an  ie  
almatier     An emerin
teor of avatar marketin ornal of Marketin
ttoior 
a    Moon   Macine a n
minlene ocial reone  to comte r
orna l of ocial  e   
ttoior
a   teer   aer    
omte r are o cial acto r in o nference
on man Fac tor in o mtin  tem 
roceein  
ttoi or    
en  M    e latet m arket
reearc  tren an lanca e in te
rowin A catot intr  Moni
ttwwwmonirocomtcnoe
a A r ti    oli  A   e
man ie of man cat ot interac tion
A tematic literatre review of te n ear
of reearc on teta e catot 
nternational o rnal of man om ter
tie   
ttoioric  
manov M  o non   
Makin converation wit ca tot
more er onal ie om ter i n
man e avior  
ttoiorc 
oma  F ale   araanna   
llan   earni n from te ark
e leverain conve rational aent in
te era of  erriva c to enance
marketin ornal of te Acaem of
Marketin cience    
mtein    n ert mark   
atot a n interactive tecn olo fo r
eronalie com mnica tion an
tranaction A nte rnation al or nal
on n ternet   
16 Marketing Review St. Gallen 4 | 2021
010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 16010-016_MRSG_01_SPT_Hildebrand_Hundertmark_2sp_mr.indd 16 17.06.21 08:3517.06.21 08:35
ResearchGate has not been able to resolve any citations for this publication.
ResearchGate has not been able to resolve any references for this publication.