COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systems

Abstract

Human-AI co-creativity involves both humans and AI collaborating on a shared creative product as partners. In a creative collaboration, interaction dynamics, such as turn-taking, contribution type, and communication, are the driving forces of the co-creative process. Therefore an interaction model is an essential component for designing effective co-creative systems. There is relatively little research about interaction design in the co-creativity field, which is reflected in a lack of focus on interaction design in many existing co-creative systems. This paper focuses on the importance of interaction design in co-creative systems with the development of the Co-Creative Framework for Interaction design (COFI) that describes the broad scope of possibilities for interaction design in co-creative systems. Researchers can use COFI for modeling interaction in co-creative systems by exploring the possible spaces of interaction.

Full text

COFI: A Framework for Modeling Interaction in Human-AI
Co-Creative Systems
Jeba Rezwana1,Mary Lou Maher1
1University of North Carolina at Charlotte, 9201 University City Blvd, Charlotte, NC 28223, US
Abstract
Human-AI co-creativity involves both humans and AI collaborating on a shared creative product as partners. In a creative
collaboration, interaction dynamics, such as turn-taking, contribution type, and communication, are the driving forces of the
co-creative process. Therefore an interaction model is an essential component for designing eective co-creative systems.
There is relatively little research about interaction design in the co-creativity eld, which is reected in a lack of focus on
interaction design in many existing co-creative systems. This paper focuses on the importance of interaction design in
co-creative systems with the development of the Co-Creative Framework for Interaction design (COFI) that describes the
broad scope of possibilities for interaction design in co-creative systems. Researchers can use COFI for modeling interaction
in co-creative systems by exploring the possible spaces of interaction.
Keywords
Co-creativity, Interaction design, Framework
1. Introduction
In human-AI co-creative systems, humans and AIs col-
laborate in a creative process as creative colleagues, and
the focus is on co-creative partnerships in contrast to cre-
ativity support tools [
1
]. Creative collaboration involves
interaction among collaborators, and the shared creative
product is more than each individual alone could achieve
[
2
]. Sonnenburg demonstrated communication as the
driving force of collaborative creativity [
3
]. Interaction
is a basic part of co-creative systems as both the human
and the AI actively participate and interact with each
other. Designing co-creative systems have many chal-
lenges due to the open-ended nature of the interaction
between the human and AI [
4
]. Bown asserted that the
success of a creative system’s collaborative role should
be further investigated concerning interaction design as
interaction plays a key role in the creative process [5].
Interaction design is oen an untended topic in the
co-creativity literature despite being a fundamental prop-
erty of co-creative systems. In recent years, researchers
designed many co-creative systems that are very intrigu-
ing and creative, yet sometimes users fail to maintain
their interest and engagement while collaborating with
the AI due to the unimpressive quality of collaboration.
An adequate interaction model dramatically improves
the quality of the collaboration and usability of a system.
Therefore, as a young eld, a holistic framework that cap-
tures the scope of interaction design is necessary. A good
starting point to investigate questions about interaction
Second Workshop on the Future of Co-Creative Systems, ICCC 2021
jrezwana@uncc.edu (J. Rezwana); m.maher@uncc.edu
(M. L. Maher)
© 2021 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
modeling is studying creative collaboration in humans
[
6
]. Understanding the factors of human collaboration
can be a tool to build the foundation for the develop-
ment of systems that can augment or enhance creativity
in humans [
7
]. The literature on computational creativ-
ity and computer-supported collaborative work (CSCW)
can also help identify interaction components related to
human-AI co-creativity.
In this paper, we present the Co-Creative Framework
for Interaction design (COFI) that denes interaction
components in a co-creation to describe the broad scope
of possibilities for interaction design in co-creative sys-
tems. These interaction components represent various
aspects of a co-creation, such as participation style, con-
tribution type, and communication between humans and
the AI. COFI is informed by the literature on human col-
laboration, CSCW, computational creativity and human-
computer co-creativity. We adopted interaction com-
ponents based on a literature review and adapted the
components to concepts relevant to co-creativity. We
argue that COFI can be used as a guide when modeling
interaction in co-creative systems as researchers can use
COFI for exploring the possible spaces of interaction.
COFI can also be useful while analyzing and interpreting
the interaction design of existing co-creative systems.
2. Related Work
In co-creative systems, humans and AI both contribute as
creative colleagues in the creative process [
1
]. Creativity
that emerges from human-AI interaction cannot be cred-
ited either to the human or to the AI alone and surpasses
both contributors’ original intentions as novel ideas arise
in the process [
8
]. Designing interaction in co-creative
systems has unique challenges due to the spontaneity
of the interaction between the human and the AI [
4
].

A co-creative AI agent needs continual adjustment and
adaptation to cope with human strategies. Mamykina et
al. argued that by understanding the factors of collabo-
rative creativity among humans, methods can be devised
to build the foundation for the development of systems
that can augment or enhance collaborative creativity [
7
].
In the eld of co-creativity, interaction design includes
various parts and pieces of the interaction dynamics be-
tween users and the AI, such as - participation style,
communication, contribution type etc. Bown argued that
the most practiced form of evaluating creative systems is
mostly theoretical and not empirically well-grounded and
suggested interaction design as a way to ground empiri-
cal evaluations of computational creativity [
9
]. Yee-King
and d’Inverno also suggested a need for integration of in-
teraction design practice into co-creativity research [
10
].
There is a lack of a framework for interaction design,
which is necessary to explain and explore the possible
interaction spaces and compare and evaluate the inter-
action design of existing co-creative systems to improve
the practice of interaction modeling.
Interaction among the individuals in collaboration
makes the process emergent and complex. For inves-
tigating human collaboration, many researchers stressed
the importance of understanding the process of interac-
tion. Fantasia et al. proposed an embodied approach of
collaboration that considers collaboration as a property
and intrinsic part of interaction processes [
11
]. Schmidt
dened CSCW as an endeavor to understand the nature
and characteristics of human collaboration to design ad-
equate computer-based collaborative technologies [
12
].
3. Co-Creative Framework for
Interaction Design (COFI)
We develop and present Co-Creative Framework for In-
teraction design (COFI) as a guiding tool that describes
the broad scope of possible spaces for interaction de-
sign in co-creative systems. This framework describes
various aspects involved in the interaction between the
human and the AI. COFI is informed by research on hu-
man collaboration, CSCW, computational creativity, and
human-computer co-creativity.
The primary categories of COFI are based on two
types of interactional sensemaking of collaboration as
described by Kellas and Trees [13]: interaction between
collaborators and interaction with the shared product.
Interaction with the shared product, in the context of co-
creative systems, describes interaction aspects related to
the creation of the creative content. Interaction between
collaborators explains how the interaction between the
human and the AI is unfolding through time which in-
cludes turn-taking, roles, timing of initiative, commu-
nication, etc. Thus, COFI characterizes relational inter-
action dynamics between the collaborators (human and
AI) as well as functional aspects of interacting with the
shared creative product. We choose the Kellas and Trees
framework for the primary categories of COFI because
they used their framework as a tool for explaining and
evaluating the interaction dynamics in human creative
collaboration in joint storytelling.
As shown in Figure 1, we further divide the two pri-
mary categories of COFI, interaction with collaborators
and interaction with the shared product, into four sub-
categories. Interaction between collaborators is divided
into collaboration style and communication style, in-
spired by research in CSCW and HCI. Interaction with
the shared product is divided into the creative process
and creative product, inspired by research in creativity
and, more specically, computational creativity. CSCW
literature discusses collaboration mechanics among hu-
mans to make eective CSCW systems, whereas creativ-
ity literature talks about creative process and product.
The interaction components are child categories of the
four main subcategories and are adapted to the context
of human-AI co-creativity. COFI is not a complete on-
tology and can continue to expand as new interaction
components emerge.
3.1. Interaction between Collaborators
(Human and AI)
This section presents components related to the relational
interaction dynamics between the human and the AI as
co-creators. As shown in Figure 1(a), interaction between
collaborators is divided into two subcategories which are
collaboration style and communication style.
3.1.1. Collaboration Style
Collaboration style is about dierent parts and pieces
of interaction between humans and the AI, related to
the nature of the co-creation. The following subsections
describe each interaction component in this category.
Participation Style: Participation style in COFI refers
to whether the collaborators can participate and con-
tribute simultaneously, or one collaborator has to wait
until the partner nishes a turn. Therefore, participation
style in COFI is categorized as parallel and turn-taking.
Categorization of participation stems from work on co-
operation in the literature [14,15].
Task Distribution: Task distribution refers to the distri-
bution of tasks among the collaborators in a co-creative
system. In COFI, there are two types of task distribution,
same task and task divided. When it is same task, there
is no division of tasks among collaborators and the col-
laborators take part in the same task. For example, in a
human-AI co-creative drawing, both co-creators do the
same task, generating the drawing. In a task-divided dis-
tribution, the main task is divided into specic sub-tasks
and the sub-tasks are distributed among the collaborators.
For example, in co-creative poetry, the user can generate

Figure 1:
Co-Creative Framework for Interaction Design (COFI): On the le (a) Components of Interaction between the
collaborators. On the right (b) Components of Interaction with the Shared Product
a poem while the AI agent can evaluate the poetry. This
component of COFI emerged from discussions of the two
interaction modes presented by Kantosalo and Toivonen:
alternating and task divided co-creativity [16].
Timing of Initiative: In a co-creative setting, the tim-
ing of both parties’ initiative taking can be scheduled
beforehand, or it can happen naturally in real-time. If
the timing of the initiative is xed in advance, in COFI,
it will be addressed as planned. If collaborators initiate
their contribution naturally without any prior plan or
xed rules, it will be addressed as spontaneous. Timing
of initiative should be chosen based on the motivation.
Spontaneous timing is suitable for increased emergent
results, whereas planned timing is suitable for systems
where users want inspiration or help in a specic way
for a particular aspect. Salvador et al. discussed timing
of initiative in their framework for evaluating groupware
for supporting collaboration [17].
3.1.2. Communication Style
Communication is a vital component in any collabora-
tion for the co-regulation between the collaborators and
helps the AI agent make the proper decision in a creative
process. Communication style includes dierent kinds
of channels to communicate between users and the AI.
Human to AI Intentional Communication: Human to
AI intentional communication channels represent the
possible ways a human agent can intentionally commu-
nicate to the AI agent to provide feedback and convey
important information to each other. Gutwin and Green-
burg proposed a framework for groupware that discusses
the mechanics of collaboration and it includes intentional
communication as a major element of collaboration me-
chanics [
18
]. In COFI, human to AI communication chan-
nel includes direct manipulation, voice, text and embod-
ied communication. The user can directly manipulate
the co-creative system by clicking buttons for giving
instructions or feedback or inputs and providing user
preferences by selecting from AI-provided options. Us-
ing the whole body or gestures for communicating with
the computer will be referred to as embodied. Accord-
ing to HCI modalities, intentional communication from
human to AI includes direct manipulation, gesture, text,
and voice [19].
Human to AI Consequential Communication: In COFI,
human to AI consequential communication channels rep-
resent the ways a co-creative AI agent can track and
collect unintentional or consequential information from
the human user, such as eye-tracking, facial expression
tracking and embodied movements. Tracking consequen-

tial details from the human is essential to perceive user
preference, user agency and engagement. Gutwin and
Greenburg reported consequential or unintentional com-
munication as a major element of collaboration mechan-
ics, in addition to intentional communication [18].
AI to Human Communication: Humans expect feed-
back and evaluation of their contribution from collabora-
tors. Therefore, if the AI agent can communicate their
status and feedback for a contribution, it would make
the co-creation more balanced as the AI agent will be
perceived as an equal partner rather than a mere tool.
The AI to user communication can include text, voice,
visuals (icons, image, animation), haptic and embodied
communication according to HCI modalities [19].
3.2. Interaction with the Shared Product
Interaction components related to the shared creative
product in a co-creative setting are discussed in this sec-
tion and illustrated in Figure 1(b). Interaction with the
shared product is divided into two subcategories, creative
contribution to the product and creative process.
3.2.1. Creative Process
Creative process characterizes the sequence of actions
that lead to a novel, and creative production [
20
]. In
COFI, there are three types of creative processes that
describe the interaction with the shared product: gen-
eration, evaluation, and denition. During a creative
generation, creative artifacts and ideas are produced in a
specic conceptual description. In a creative evaluation,
produced ideas, artifacts or concepts get assessed to be
more rened and appropriate for the creative objective.
In a creative denition process, collaborators determine
and prepare the creative conceptual space. For example,
a co-creative AI agent can dene the attributes of a c-
tional character before a writer starts to write about the
character. The basis of this categorization is the work
of Kantosalo et al. that denes the roles of the AI as
generator, evaluator, and concept dener [16].
3.2.2. Creative Product
The creative product is the idea or concept that is being
created. Creative product has two interaction compo-
nents, contribution type and contribution similarity.
Contribution Type: In a co-creation, an individual can
contribute in dierent ways to the shared product. Co-
creators can generate new elements, extend the existing
contribution, modify or rene the existing contribution.
The primary contribution types in COFI are: ’create new’,
’extend’, ’transform’ and ’rene’. ’Extend’ refers to ex-
tending the contribution of the partner or adding on to
the partner’s contribution. Generating something new
or creating new objects is represented by ’create new’,
whereas ’transform’ conveys turning the partner’s con-
tribution into something totally dierent. ’Rene’ is eval-
uating and correcting the partner’s contributions with
a similar type of contribution. Contribution types are
adopted and adapted from Boden’s categories of computa-
tional creativity based on dierent types of contribution:
combinatorial, exploratory, and transformational [21].
Contribution Similarity: Contribution similarity refers
to the degree of similarity or association in terms of
the contribution compared to the partner’s contribution.
Both convergent and divergent exploration have value in
a creative process. Basadur et al. asserted that divergent
thinking is related to the ideation phase and convergent
thinking is related to the evaluation phase [22].
4. Conclusions
As a growing eld, human-computer co-creativity lacks
signicant research on interaction models and their impli-
cations in co-creative systems. Human-AI co-creativity
research needs a holistic framework that captures aspects
and components of interaction to design eective co-
creative systems. In recent years, a few frameworks have
been developed about interaction design in co-creative
systems. However, they lack a focus on interaction com-
ponents related to the interaction between collaborators
as distinct from interaction components related to the
shared product. We develop and describe COFI as a new
framework to provide a way for researchers to explore
the design space of interaction for a specic system. COFI
will provide useful guidelines for interaction modeling
while developing co-creative systems. COFI can also be
benecial while investigating and interpreting the in-
teraction design of existing co-creative systems. As a
framework, COFI is expandable as other interaction com-
ponents can be added to it in the future. By establishing
COFI, we can look for the relationship between dierent
interaction designs in co-creative systems and dierent
creative outcomes. COFI can be also used to develop
co-creative systems that can improve user engagement
with eective collaboration strategies through adequate
human-AI interaction.
References
[1]
N. M. Davis, Human-computer co-creativity: Blend-
ing human and computational creativity, in: Ninth
Articial Intelligence and Interactive Digital Enter-
tainment Conference, 2013.
[2]
R. K. Sawyer, S. DeZutter, Distributed creativity:
How collective creations emerge from collabora-
tion., Psychology of aesthetics, creativity, and the
arts 3 (2009) 81.
[3]
F. K. Sonnenberg, Strategies for creativity, Journal
of Business Strategy (1991).
[4]
N. Davis, C.-P. Hsiao, K. Yashraj Singh, L. Li,
B. Magerko, Empirically studying participa-
tory sense-making in abstract drawing with a co-

creative cognitive agent, in: Proceedings of the
21st International Conference on Intelligent User
Interfaces, 2016, pp. 196–207.
[5]
O. Bown, Player responses to a live algorithm: Con-
ceptualising computational creativity without re-
course to human comparisons?, in: ICCC, 2015, pp.
126–133.
[6]
N. Davis, C.-P. Hsiao, Y. Popova, B. Magerko, An
enactive model of creativity for computational col-
laboration and co-creation, in: Creativity in the
Digital Age, Springer, 2015, pp. 109–133.
[7]
L. Mamykina, L. Candy, E. Edmonds, Collaborative
creativity, Communications of the ACM 45 (2002)
96–99.
[8]
A. Liapis, G. N. Yannakakis, J. Togelius, Computa-
tional game creativity, ICCC, 2014.
[9]
O. Bown, Empirically grounding the evaluation of
creative systems: Incorporating interaction design.,
in: ICCC, 2014, pp. 112–119.
[10]
M. Yee-King, M. d’Inverno, Experience driven de-
sign of creative systems (2016).
[11]
V. Fantasia, H. De Jaegher, A. Fasulo, We can work
it out: an enactive look at cooperation, Frontiers
in psychology 5 (2014) 874.
[12]
K. Schmidt, Cooperative work and coordinative
practices, in: Cooperative Work and Coordinative
Practices, Springer, 2008, pp. 3–27.
[13]
J. K. Kellas, A. R. Trees, Rating interactional sense-
making in the process of joint storytelling, The
sourcebook of nonverbal measures: Going beyond
words (2005) 281.
[14]
D. W. Johnson, R. T. Johnson, New developments
in social interdependence theory, Genetic, social,
and general psychology monographs 131 (2005)
285–358.
[15]
T. Liu, H. Saito, M. Oi, Role of the right inferior
frontal gyrus in turn-based cooperation and com-
petition: a near-infrared spectroscopy study, Brain
and cognition 99 (2015) 17–23.
[16]
A. Kantosalo, H. Toivonen, Modes for creative
human-computer collaboration: Alternating and
task-divided co-creativity, in: Proceedings of the
seventh international conference on computational
creativity, 2016, pp. 77–84.
[17]
T. Salvador, J. Scholtz, J. Larson, The denver model
for groupware design, ACM SIGCHI Bulletin 28
(1996) 52–58.
[18]
C. Gutwin, S. Greenberg, M. Roseman, Workspace
awareness in real-time distributed groupware:
Framework, widgets, and evaluation, in: People
and Computers XI, Springer, 1996, pp. 281–298.
[19]
L. Nigay, Design space for multimodal interaction,
in: Building the Information Society, Springer, 2004,
pp. 403–408.
[20]
T. I. Lubart, Models of the creative process: Past,
present and future, Creativity research journal 13
(2001) 295–308.
[21]
M. A. Boden, Creativity and articial intelligence,
Articial Intelligence 103 (1998) 347–356.
[22]
M. Basadur, P. A. Hausdorf, Measuring divergent
thinking attitudes related to creative problem solv-
ing and innovation management, Creativity Re-
search Journal 9 (1996) 21–32.