ArticlePDF Available

Abstract and Figures

The goal of this study was to examine the effectiveness of brainwriting in an industrial setting. Research has demonstrated that group interaction can inhibit idea generation. Written exchanges of ideas in groups have been found to be an effective way to increase idea generation. To our knowledge, no study has examined the potential of brainwriting for group idea generation in work settings or the impact of different sequences of group and individual idea generation. Participants in a high-technology company participated in two brainwriting studies. In one study, participants generated ideas either first alone and then in a group or in the reverse order. In a second study, participants either generated as a group during the entire session or alternated individual ideation with a periodic review of the group's ideas. In the first study, participants who generated ideas first as a group and then as individuals performed best. In the first session, group writing also tended to lead to more ideas than did individual writing. In the second study, participants with periodic reviews performed best. The results suggest that alternation in individual and group brainwriting can enhance the number of ideas generated. The group-to-alone sequence is also beneficial since it allows group members to build on shared ideas. This research indicates that collaborative idea sessions can be beneficial in work sessions if the brainwriting paradigm is used with an appropriate alternation of group ideation or review sessions with individual idea generation sessions. © 2015, Human Factors and Ergonomics Society.
Content may be subject to copyright.
Greetings, and thank you for publishing with SAGE Publications. We have prepared this page proof for your review.
Please respond to each of the below queries by digitally marking this PDF and sending the marked PDF back.
Please use only the icons shown below for your mark-up, and do not use “sticky notes” or other “pop-up” comments.
Please use: Please do NOT use:
Help with Digital Mark-up
1. Do you have Adobe Reader? Quick, safe, free download is available at
2. To access the recommended tools, please click “Comment” in upper right of Adobe Reader, then
“Drawing Markups.”
3. Using tools other than those recommended may cause data loss.
4. Some mark-up examples are shown below.
Please note that sending corrections in any form other than the marked-up PDF can signicantly
delay the processing of your corrections.
These proofs is an opportunity to correct any errors that may have been introduced during the
the production process not an opportunity for rewriting. Your article has been peer reviewed and
accepted, so any substantial rewriting will need to be approved by the journal editor, delaying
publication significantly. In addition to requiring journal editor approval, a fee may be
charged for extensive rewriting not due to SAGE error or request.
are [delete]
Please assist us by clarifying the following queries:
No Query
Journal title: Human Factors
Article Number: 570374
1 Please confirm that all author information, including names, affiliations, sequence, and contact details, is
2 Please review the entire document for typographical errors, mathematical errors, and any other necessary
corrections; check headings, tables, and figures.
3 Please confirm you have reviewed this proof to your satisfaction and understand this is your final opportunity
for review prior to publication.
4 We have two different subtitles for the article: “Exploratory Studies” and “Evidence From Two Exploratory
Studies.” Please clarify which subtitle is correct.
5 In the sentence beginning “The increased sharing of ideas…,” citation for Brown & Paulus, 2007, is missing
a corresponding reference. Please correct citation or provide complete reference details.
6 In the sentence beginning “Similarly, detailed observation…,” citation for Dunbar (2007) is missing a
corresponding reference. Please correct citation or provide complete reference details.
7 Under Study 2, should “Hypothesis 1” be changed to “Hypothesis 3,” since Hypotheses 1 and 2 were
introduced under Study 1?
Objective: The goal of this study was to examine
the effectiveness of brainwriting in an industrial setting.
Background: Research has demonstrated that group
interaction can inhibit idea generation. Written exchanges
of ideas in groups have been found to be an effective way
to increase idea generation. To our knowledge, no study
has examined the potential of brainwriting for group idea
generation in work settings or the impact of different
sequences of group and individual idea generation.
Method: Participants in a high-technology com-
pany participated in two brainwriting studies. In one
study, participants generated ideas either first alone
and then in a group or in the reverse order. In a second
study, participants either generated as a group during
the entire session or alternated individual ideation with
a periodic review of the group’s ideas.
Results: In the first study, participants who gen-
erated ideas first as a group and then as individuals
performed best. In the first session, group writing
also tended to lead to more ideas than did individual
writing. In the second study, participants with periodic
reviews performed best.
Conclusion: The results suggest that alternation in
individual and group brainwriting can enhance the num-
ber of ideas generated. The group-to-alone sequence is
also beneficial since it allows group members to build
on shared ideas.
Application: This research indicates that collab-
orative idea sessions can be beneficial in work sessions
if the brainwriting paradigm is used with an appropriate
alternation of group ideation or review sessions with
individual idea generation sessions.
Keywords: creativity, brainstorming, brainwriting,
idea selection, collaborative innovation, groups
Organizations and companies are constantly
engaged in processes designed to develop new
methods, processes, and products aimed at
building and sustaining their competitive edge.
Often the ideas that are the basis for innova-
tions are developed in group settings, such as
team meetings or formal brainstorming ses-
sions (Dunbar, 1997; Petre, 2004). Information
exchange (Carmeli, Gelbard, & Reiter-Palmon,
2013) and knowledge integration (Carmeli &
Azerual, 2009; Collins & Smith, 2006) are obvi-
ously important components of this process.
Although group interaction can be a source of
creative ideas, a number of features of the inter-
action process tend to limit the effectiveness of
the group ideation process. In face-to-face set-
tings, the opportunity to fully share information
and knowledge is limited by the fact that only
one person can express his or her ideas at one
time. While waiting one’s turn to share ideas, a
person may forget what he or she meant to say or
get distracted from one’s own ideas by the shar-
ing process. There may be rather uneven partici-
pation as some individuals may dominate the
discussion. These and other problems have been
cited as reasons for the fact that face-to-face
groups typically generate fewer ideas than nom-
inal groups (the pooled number of nonredundant
ideas from the same number of individual per-
formers; see Nijstad & Stroebe, 2006; Paulus &
Coskun, 2012). However, most of the relevant
studies have been done in laboratory conditions
with university students using tasks that may not
have much meaning to the students or relevance
to their expertise. It is possible that in work set-
tings, groups may be more effective if they are
engaged in discussion of issues of personal
importance. Only limited studies of this type
have been done, but their findings indicate that
even in those contexts, face-to-face groups tend
XXX10.1177/0018720815570374Human FactorsAsynchronous Brainstorming
Address correspondence to Paul B. Paulus, Department of
Psychology, University of Texas at Arlington, Box 190047,
Arlington, TX 76019-0528, USA; e-mail:
Asynchronous Brainstorming in an Industrial
Setting: Exploratory Studies[AQ: 1]
Paul B. Paulus, Runa M. Korde, Jubilee J. Dickson, University of Texas at
Arlington, Abraham Carmeli, Tel Aviv University, Tel Aviv, Israel, and Ravit
Cohen-Meitar, Tmurot Ltd., Tel Aviv, Israel[AQ: 2][AQ: 3][AQ: 4]
Vol. XX, No. X, Month XXXX, pp. 1 –19
DOI: 10.1177/0018720815570374
Copyright © 2015, Human Factors and Ergonomics Society.
2 Month XXXX - Human Factors
to perform more poorly than nominal groups
(e.g., Dunnette, Campbell, & Jaastad, 1963;
Paulus, Larey, & Ortega, 1995).
The negative results for brainstorming are sur-
prising in part because group interaction should
be a source of cognitive stimulation. Ideas from
others should challenge participants (Amabile,
1996) and stimulate them to think of categories
of ideas they might not have considered or pro-
vide associations for additional ideas (Nijstad &
Stroebe, 2006; Paulus & Brown, 2003, 2007).
Furthermore, group interaction provides an
opportunity to build on the ideas of others (Kohn,
Paulus, & Choi, 2011) and facilitates help seek-
ing, help giving, and reflective reframing (Har-
gadon & Bechky, 2006) as well as cultivating a
reflection in conversation, all vital to the ideation
process (Carmeli, Dutton, & Hardin, in press). It
appears that longer sessions can allow interactive
groups to “catch up” to nominal groups (Nijstad,
van Vianen, Stroebe, & Lodewijkx, 2004), but to
the best of our knowledge, studies have not
shown that such face-to-face brainstorming
groups can exceed the performance of nominal
groups (a form of synergy; Larson, 2010).
Although most studies have measured number of
ideas, those that have measured rated novelty
and utility show that the average quality of ideas
is typically similar for both interactive and nomi-
nal groups (e.g., Diehl & Stroebe, 1987). How-
ever, since nominal groups tend to generate more
ideas, they also generate more good ideas (novel
and useful; Paulus, Kohn, & Arditti, 2011).
A number of researchers have examined the
possibility that paradigms that allow for a more
efficient exchange of ideas in groups can over-
come the limitations of face-to-face verbal
brainstorming. One approach involves the use of
electronic brainstorming, in which participants
exchange ideas by means of computer systems.
This technique can overcome the typical pro-
ductivity loss found in verbal interactions (Den-
nis & Williams, 2003). Thus groups can perform
as well as nominal groups or even better in the
case of groups larger than eight (Dennis & Wil-
liams, 2005; DeRosa, Smith, & Hantula, 2007).
The beneficial effects of larger groups in elec-
tronic brainstorming appears to derive from the
fact that such groups increase the number of
ideas to which participants are exposed, but the
increase in number of ideas generated per person
is rather small (Paulus, Kohn, Arditti, & Korde,
2013). However, a study in an industrial setting
that allowed participants 4 days to generate their
ideas showed that the performance of individu-
als was comparable to that of a large group
(Dornburg, Stevens, Hendrickson, & Davidson,
2009). Moreover, the quality of the ideas was
higher for the individual performers. Therefore,
the findings regarding the effectiveness of elec-
tronic brainstorming for the generation of cre-
ative ideas are inconclusive, and this question
remains open (see also Barki & Pinsonneault,
2001; Paulus et al., 2013).
Another approach to facilitating idea genera-
tion in groups is to exchange ideas on pieces of
paper. This approach is called brainwriting
(Michinov, 2012; Van Gundy, 1983) and can
take various forms, such as sharing ideas among
group members in a rotational manner (passing
slips from one person to another as they are gen-
erated) or in a gallery approach (writing ideas on
sheets of paper and then allowing the group to
view them; Linsey et al., 2011). Research has
shown that brainwriting has the potential for
increasing group productivity in comparison to
face-to-face groups (Heslin, 2009; Van Gundy,
Brainwriting, like electronic brainstorming,
avoids the production blocking of face-to-face
brainstorming and allows group members to
share ideas as they occur. The increased sharing
of ideas allows for greater stimulation of ideas
as group members read each other’s ideas
(Brown & Paulus, 2007[AQ: 5]). Furthermore,
the brainwriting paradigm increases the likeli-
hood that individuals will carefully attend to the
shared ideas since reading ideas of others is part
of the procedure (but optional in most electronic
brainstorming systems). Attention to the ideas of
others is of course a critical factor for observing
stimulation effects in groups (Brown & Paulus,
2002; Michinov, Jamet, Métayer, & Le Hénaff,
in press; Paulus & Brown, 2007). For example,
Paulus and Yang (2000) found that brainwriting
groups of four generated more ideas than did
nominal groups. This difference was even more
evident in a subsequent session in which ideas
were generated alone in both conditions. Thus,
the second session allowed group members to
Asynchronous BrAinstorming
build on the shared ideas from the first session in
generating even more ideas. A more recent study
showed that the fact that the nominal group
members in the Paulus and Yang study wrote
ideas on a sheet of paper instead of individual
pieces of paper used by the groups may have
contributed to the enhanced performance of
groups (Goldenberg, Larson, & Wiley, 2013).
When both interactive and nominal groups used
paper slips, they found no difference in the num-
ber of ideas generated. However, since the inter-
active groups performed the same as the nomi-
nal groups, it is still a positive result in compari-
son to the typical results in verbal groups.
Paulus and Yang’s (2000) study suggests that
variation in group and individual brainstorming
may be beneficial. In fact, Osborn (1957) pro-
posed that alternation in group and individual
brainstorming might be an optimal procedure.
This notion makes sense from a variety of per-
spectives. In practice, individuals in the work-
place are likely to think ahead of time about the
topic to be brainstormed. After a group brain-
storming session, they may think of additional
ideas. However, there are few systematic studies
of brainstorming sequence, and the results are
rather mixed (cf. Baruah & Paulus, 2009). For
example, Baruah and Paulus (2008) found that
for highly trained participants in a verbal brain-
storming setting, the alone-to-group sequence
led to more ideas than did the group-to-alone
one. Other studies have suggested that the
group-to-alone sequence can be beneficial (Dun-
nette et al., 1963, Nagasundaram & Dennis,
1993; Paulus, Dugosh, Dzindolet, Coskun, &
Putman, 2002). Research findings suggest that
the group-to-alone sequence can be quite benefi-
cial in brainwriting (Paulus & Yang, 2000). This
result may be in part because the group sharing
process is more effective in brainwriting, and
thus positive carryover effects of cognitive stim-
ulation are more likely (Brown & Paulus, 2007).
In verbal brainstorming, alone brainstorming is
more effective, and this rate of productivity may
carry over into a subsequent group brainstorm-
ing session (Baruah & Paulus, 2008).
There have been only a few controlled studies
of brainstorming in work settings (Dornburg
et al., 2009; Dunnette et al., 1963; Paulus et al.,
1995), and no published study has dealt with
brainwriting in this type of setting. There have
been a number of other researchers who have
examined collaborative creativity in the broader
context of teamwork in organizations. Reviews
of the team creativity and innovation literature
have suggested a number of factors related to
increased innovation in teams, such as job diver-
sity and organizational support (Hülsheger,
Anderson, & Salgado, 2009; Paulus, Dzindolet,
& Kohn, 2011). However, practically all of the
team innovation studies have used self- or other
reports as the basis for their outcome measures.
There have been no systematic studies that have
used performance measures in teamwork con-
text and have compared these results with appro-
priate control groups for degree of creative syn-
ergy. That is, these studies have not demon-
strated that work teams are more innovative or
creative than similar-sized collections of indi-
vidual performers.
The clearest evidence of collaborative syn-
ergy has come from research in problem solving,
decision making, and group learning (Larson,
2010). Few of these studies have been done with
actual teams in work contexts, but some of these
have demonstrated synergistic or positive effects
of collaboration in work settings or with partici-
pants from work settings. For example, Chan and
Schunn (2014) examined brainstorming sessions
of a professional design team and explored the
role of analogies in leading to novel concepts. In
a similar study of a design problem with engi-
neering students, Chan et al. (2011) found that
exposure of more distant analogies (from patent
domains that differed from the focal problem)
increased the number of novel concepts relative
to a control condition. In a study of collaborative
memory, pairs of expert pilots demonstrated
facilitation of performance relative to nominal
dyads, but novice pilots and nonpilots showed
collaborative inhibition (Meade, Nokes, & Mor-
row, 2009). Similar results were found with a
problem-solving task (Nokes-Malach, Meade, &
Morrow, 2012). Mullins, Rummel, and Spada
(2011) found that collaboration in a learning
environment enhanced acquisition of conceptual
Research on team performance in work set-
tings is difficult because in our experience and that
reported informally by many other researchers,
4 Month XXXX - Human Factors
both management and employees tend to resist the
intrusion of time-intensive research in the work-
place since it takes away from worker productivity
and thus can be quite costly. For that reason, sur-
vey research seems to dominate in the literature on
teams. Yet given the importance of innovation for
organizational competitiveness, it is important that
performance-based research be done in organiza-
tional and industrial settings. For example, in a
case study of 12 engineering firms, Petre (2004)
noted that brainstorming played an important role
in high-performing teams. Thus, understanding
effective brainstorming in an industrial setting can
potentially increase the effectiveness of teams that
are concerned with innovation. Similarly, detailed
observation of some high-level scientific teams
led Dunbar (2007[AQ: 6]) to conclude the quality
of the idea exchange process in laboratory meet-
ings was critical in determining which teams made
the most significant discoveries.
We had an opportunity to examine brainwrit-
ing in a work setting that consisted of 80 people
working in research and development for a mul-
tinational firm. The manager of this group felt
that an increase in number of employees had not
significantly increased the productivity of the
unit. So he was most interested in having us
examine ways by which his teams could func-
tion more effectively. We spent an entire day
with the employees, conducting two studies and
then providing them feedback about how to
enhance team performance and innovation in a
lecture/discussion format. Although this was an
exciting setting for research, we had a limited
sample size. Therefore, obtaining statistical sig-
nificance would be a challenge. However, we
felt that the trends we would obtain would be of
practical and theoretical significance.
In Study 1, we asked participants to write
ideas on slips of paper in groups of three. In
one condition, the group members brainstormed
alone first and then as an interactive group. In
the other condition, participants brainstormed
as a group and then alone. Our primary interest
was in the average number of ideas generated.
However, we also assessed the originality and
utility of the ideas but did not expect differences
in quality since they are not typically found for
group and individual brainstorming variations.
Because both the alone and group conditions
involved the use of individual slips of paper,
we avoided the confounding factor of different
methods of writing ideas as in the Paulus and
Yang (2000) study.
After the performance session, the groups
were asked to pick the three best ideas. Groups
and individuals in laboratory studies typically
have a difficult time picking the best or most
original ideas, and the average novelty of the
selected ideas tends not to be higher than the
average of all of the ideas generated (Putman &
Paulus, 2009; Rietzschel, Nijstad, & Stroebe,
2006). Hence, this study would enable us to
determine whether this result would also be the
case in an industrial setting.
The first phase of the study allowed us to
examine whether the group idea exchange was
more beneficial for generating ideas than the
alone condition in an industrial setting, as was
found by Paulus and Yang (2000). The demon-
stration of such group creative synergy is based
on the anticipated reduction in production block-
ing and the expectation that group exchange of
ideas can provide interpersonal stimulation
among the group members in which ideas from
others may suggest additional categories of
ideas or generate associations that can be the
basis for additional ideas. This would be consis-
tent with the cognitive models of group brain-
storming (Nijstad & Stroebe, 2006; Paulus &
Brown, 2003, 2007). The demonstration of syn-
ergy would require that the benefits of group
stimulation would be greater than the potential
negative effects of the time demands for reading
others’ ideas and their potential to interfere with
the individual flow of idea generation.
We also predicted that the group-to-alone
sequence would lead to more ideas and more
good ideas than the alone-to-group condition for
a variety of reasons. If in the first phase the
group condition indeed leads to the stimulation
of more ideas, this stimulation should carry over
to the second session when participants are able
to build on these ideas and the related cognitive
associations. Furthermore, the higher level of
performance in Session 1 of the group-to-alone
condition may carry over to the alone session (a
form of entrainment; Kelly and McGrath, 1985).
Asynchronous BrAinstorming
Similarly, group brainwriting is somewhat con-
straining since participants have to read ideas as
well as generate them. Once they are free of that
demand in the alone session, there may be
increased motivation to fully express their own
ideas. Even though the group session may be a
bit constraining, group members tend to feel
more positive about group ideation more than
individual ideation (cf. Paulus et al., 1995). This
positive feeling can carry over to the subsequent
alone session and thus be another factor in
enhancing performance.
We did not predict differences in degree of
novelty and utility since most previous studies
of group and individual brainstorming have not
revealed differences for these measures. How-
ever, the condition that generated the most ideas
was also expected to generate the maximum
number of good ideas (those high in both nov-
elty and utility).
Thus, we suggest the following hypotheses:
Hypothesis 1: Group brainwriting is more ben-
eficial for generating ideas and good ideas
than is solitary brainwriting.
Hypothesis 2: In a brainwriting context, the
group-to-alone condition would yield the
best performance in terms of number of
ideas and number of good ideas.
Fifty-seven of the employees of a high-tech-
nology company in Israel participated in this
study. One workday was dedicated to the con-
duct of this and a second study. The entire group
met for an introduction of the plans for the
day by the manager and its importance for the
organization. Then one member of the research
team gave the general instructions to the group
about the nature of the study and the task pro-
cedures. They were told that at the end of the
brainstorming activities, each group would be
asked to share their best two ideas from the two
different sessions and that the best three ideas
would be awarded prizes. The manager actually
decided what he thought were the best ideas and
awarded the prizes at the end of the discussion.
After the initial instructions, two experiments
were conducted with a break of about 40 min
between them. During part of this break, partici-
pants completed an unrelated survey.
The participants in Study 1 were arbitrarily
divided into two different groups and sent to two
separate rooms for the two conditions. Partici-
pants worked in groups of two or three. Both
conditions had 10 groups each. There was one
group of two in the alone-to-group condition,
and there were two groups of two in the group-
to-alone condition. Each group was seated at a
separate table. Participants were asked to gener-
ate ideas about how to make an excellent, sig-
nificant, and effective team. This topic and the
one in Study 2 were generated by a leadership
team as being of special interest to its unit. The
participants were instructed,
We would like you to generate ideas about
how to make an excellent, significant, and
effective team. Any ideas on this issue are
encouraged. This can include ideas about
new employee integration, how to make
an excellent team, how to make the team
recognized, having fun at work, and being
an efficient team. These are just some of
the areas you can consider.
There were two sessions of 10 min each. Par-
ticipants were provided with slips of paper on
which to write their ideas. They were instructed
to write their ideas briefly on the slips of paper
and not to worry about spelling or grammar.
They were instructed to write only one idea on
each slip. They could write their ideas in either
English or Hebrew. In the alone-to-group condi-
tion, participants wrote ideas on slips of one
color for 10 min individually and placed them in
a pile in front of them and were instructed not to
share them. In the second session, they wrote
ideas on a different-color slip and were asked to
pass them on to the person on their right. They
were instructed to read the ideas as they were
passed around and place ideas in the center of
the table when their own idea came back to
them. They were instructed that if they could not
think a new idea at some point, they could pass
along some of the slips they generated in the first
session. This was designed to make this condi-
tion a bit more “natural” since in “real groups,”
individuals would tend to share ideas they had
6 Month XXXX - Human Factors
thought of before the session as well as during
the session. However, they were encouraged to
keep generating new ideas until they were told
to stop. In the group-to-alone condition, this
sequence was reversed. Participants were told to
share their ideas in the first session and not to
share them in the second session. The detailed
instructions for the group session in the group-
to-alone condition were as follows:
You are about to participate in a study
examining idea generation. In a minute
you will be given a topic. Your job is
to list as many ideas as possible for this
topic. These ideas can be as short as a
few words. Do not worry about perfect
spelling or grammar. You will be writing
your ideas on small slips of paper. Please
write each idea on a different slip and pass
the slips to the person on your right once
you finish writing your idea. You will be
receiving the ideas from the person on
your left. If you receive this idea while
writing your own, finish writing your idea
and then read the one you just received.
When you are done reading the idea, pass
it on to the next person. Once your own
idea comes back to you, place the slip of
paper in the center of the table.
The detailed instructions for the subsequent
alone session were as follows:
You can stop now. For the next phase,
you will continue to generate ideas on the
same topic. However, now we would like
you not to pass the slips to the person on
your right but simply place it in a pile in
front of you once you finish writing your
idea. Follow the same brainstorming rules
as before as you generate ideas on your
own. This time please write the ideas only
on the blue slips of paper. Try to keep gen-
erating ideas until you hear “Stop.”
The detailed instructions for the alone-to-
group condition were the same with slight word-
ing changes to accommodate the reverse order.
Participants were given the typical brainstorm-
ing instructions that criticism was ruled out, that
freewheeling was welcome, that quantity was
wanted, and to combine and improve on the
ideas already generated. Then they were pro-
vided with the topic for the brainstorming ses-
sion. After the brainstorming session, partici-
pants were provided 15 to 20 min to look over
all of the ideas and place checkmarks on the
ideas they thought were the best ones. Then they
were asked to collectively decide on the best
three ideas. Afterward, they were asked to com-
plete a short questionnaire to tap their evalua-
tions of their experience during the brainstorm-
ing session. These items were rated on 5-point
scales. The items tapped motivation, benefit of
the sharing process, and ratings of number and
quality of ideas generated by the participant. For
example, “How motivated were you?” rated
from not at all (1) to a very large extent (5), and
“Reviewing your group members’ ideas helped
you generate more ideas,” rated from not at all
(1) to a very large extent (5), were two of the
The primary measure of interest was the
number of ideas generated since that has been
the main focus of past studies. The ideas written
on the slips tended to include multiple elements
and often multiple lines of text. No ideas were
exactly alike, so we counted all the ideas gener-
ated. Since the group sizes were uneven across
both conditions, average number of ideas was
used as the main dependent variable instead of
total number of ideas. We also had the ideas
rated on a five-point scale (with 5 being high)
for originality and utility by two independent
raters. The coders were presented files to code
that did not identify the conditions.
Both raters coded 25% of the data, and scores
that were within one point of each other were
considered as indicators of agreement. The
resulting intraclass correlation (ICC) value for
single measures was .612 and the Cronbach’s
alpha was .760 for originality. For utility, the
ICC single-measures value was .624 and the
Cronbach’s alpha was .769. One of the raters
then coded the remaining 75% of the data for
both originality and utility.
Average ideas per person per group. Numer-
ically more ideas were generated by group
Asynchronous BrAinstorming
members in the first phase of the group-to-alone
condition than in the first phase of the alone-to-
group condition, although this effect was not
significant, F(1, 18) = 2.022, p = .172, ηp
2 =
.101. The group-to-alone condition yielded more
ideas than the alone-to-group condition overall,
an effect that approached significance, F(1,
18) = 4.315, p = .052, ηp
2 = .193. There was also
a main effect of phases (Phase 1 > Phase 2), F(1,
18) = 11.449, p = .003, ηp
2 = .389, with fewer
ideas generated in the second session. There was
also a significant interaction between condition
and phase number, F(1, 18) = 5.181, p = .035,
2 = .224 (see Table 1). The interaction effect
reflects the decline in the group brainwriting
phase of the alone-to-group condition. In the
group-to-alone condition, the group started at a
high level, and this level was maintained during
the alone phase.
Average novelty and average utility. The
data were also analyzed for differences in aver-
age novelty and utility ratings. There was very
little difference in the ratings of novelty between
both conditions, F(1, 18) = .331, p = .572, ηp
2 =
.018, as well as over time, F(1, 18) = 1.983, p =
.176, ηp
2 = .099. The interaction between the
condition and phases was not significant, F(1,
18) = .682, p = .420, ηp
2 = .037. In line with
these results, there was no significant difference
in the average novelty of ideas generated while
brainstorming alone or as a group, t(19) = .805,
p = .431. The average novelty of the best ideas
selected in both conditions was very similar,
t(19) = −1.395, p = .180. Additional analyses
showed that the three ideas selected as the best
ideas did not have higher average novelty as
compared to the pool of ideas from which they
were selected, t(19) = .438, p = .666. The same
analysis was also performed for each condition
separately and revealed no significant differ-
ences: alone to group, t(9) = −.549, p = .596;
group to alone, t(9) = 1.087, p = .305.
Similar results were obtained for utility of the
ideas: There were no significant main effects of
condition, F(1, 18) = .491, p = .493, ηp
2 = .027,
or time, F(1, 18) = .928, p = .348, ηp
2 = .049, and
the interaction effect was not significant, F(1,
18) = 3.321, p = .085, ηp
2 = .156. The ideas gen-
erated in the alone phases were not significantly
different as compared to those generated in the
group phases with regard to their utility, t(19) =
1.826, p = .084. The average utility of the best
ideas was not higher than the utility of the pool
of ideas from which they were selected, t(19) =
.738, p = .470. The same analysis was also
performed for each condition separately and
TABLE 1: Study 1: Quantity, Novelty, and Utility of Brainstorming Ideas by Condition and Phase
Condition Average Quantity Novelty Utility Number of Good Ideas
Alone to group
Phase 1 3.95 (1.764) 2.440 (0.394) 3.199 (0.183) 4.10 (2.726)
Phase 2 2.417 (1.399) 2.496 (0.628) 2.879 (0.361) 1.80 (1.751)
Best ideas 2.367 (0.618) 3.167 (0.324)
Overall 3.183 (1.516) 2.468 (0.486) 3.039 (0.213) 5.90 (2.726)
Group to alone
Phase 1 5.350 (2.566) 2.477 (0.517) 3.061 (0.353) 5.40 (2.716)
Phase 2 5.050 (2.944) 2.693 (0.433) 3.160 (0.428) 6.50 (4.649)
Best ideas 2.808 (0.788) 2.875 (1.208)
Overall 5.200 (2.670) 2.585 (0.486) 3.110 (0.241) 11.90 (6.523)
Phase 1s combined 4.650 (2.260) 2.458 (0.448) 3.130 (0.282) 4.75 (2.731)
Phase 2s combined 3.733 (2.619) 2.595 (0.539) 3.019 (0.412) 4.15 (4.184)
Alone phases 4.500 (2.429) 2.567 (0.429) 3.180 (0.321) 5.30 (3.908)
Group phases 3.883 (2.512) 2.486 (0.560) 2.970 (0.360) 3.55 (2.837)
Note. Standard deviations are shown in parentheses. There were 10 groups in both the conditions.
8 Month XXXX - Human Factors
revealed no significant differences: alone to
group, t(9) = −1.447, p = .182; group to alone,
t(9) = .638, p = .540.
Number of good ideas. We used a score of 3
or more on both originality and utility as a crite-
rion for good ideas because that captured a rea-
sonable number of ideas. This cutoff of a score
of 3 or higher has been used in previous studies
(cf. Paulus et al., 2011). The number of good
ideas was counted across conditions and phases,
and a mixed ANOVA was used to examine the
differences. There was a significant effect of
condition, F(1, 18) = 7.293, p = .015, ηp
2 = .286.
A larger number of good ideas was generated in
the group-to-alone condition (M = 5.950) as
compared to the alone-to-group condition (M =
2.950). There was no main effect of phases, F(1,
18) = .497, p = .490, ηp
2 = .027, but a trend
toward interaction effect of phase and condition,
F(1, 18) = 3.986, p = .061, ηp
2 = .181. This effect
reflected the fact that there was a decline in
number of good ideas in the group phase of the
alone-to-group condition but an increase in
number of good ideas in the alone phase of the
group-to-alone condition. We also examined the
number of ideas with scores of 4 or higher on
originality and utility. However, the number of
such ideas was quite small, and thus no effects
were obtained.
Questionnaire data. Responses to the post-
study questionnaire items were analyzed using t
tests. The questionnaire was aimed at gaining an
understanding of the participants’ perception of
their performance. These performance criteria
were feelings of competition, effort expended,
motivation, how carefully they read others’
ideas, effects of exposure to others’ ideas, qual-
ity of their ideas, and quantity of their ideas.
Two participants did not complete the question-
naire. The means and standard deviations for
these items are reported in Table 2. There were
no differences between the conditions with
regard to how competitive the participants felt,
t(16) = −1.158, p = .264, or how much effort
they thought they expended, t(16) = −1.437, p =
.170. For all the remaining items, participants in
the group-to-alone condition reported higher
values than those in the alone-to-group condi-
tion. Participants in the group-to-alone condi-
tion reported higher motivation, t(16) = −2.267,
p = .038, and having read the ideas of their group
members more carefully, t(16) = −2.372, p =
.031. They felt that seeing others’ ideas helped
them generate more ideas, t(16) = −2.588, p =
.020. However, they did not feel that seeing
those ideas helped them generate better ideas,
t(16) = −1.948, p = .069. When asked about their
productivity, participants in the group-to-alone
condition reported generating more high-quality
ideas, t(16) = −2.517, p = .023; however, there
was no significant difference in the number of
ideas the participants thought they generated,
t(16) = −1.960, p = .068.
A major finding of interest is the trend to supe-
rior performance of the group-to-alone condition
in comparison to the alone-to-group condition in
terms of average number of ideas. This finding
is consistent with prior research indicating that
the group-to-alone condition may be best (Paulus
& Yang, 2000). In the first session, the average
number of ideas generated in groups was 37%
higher than in the alone condition. Although this
finding was not significant, probably in part due
to the limited sample size, this effect is practi-
cally important because it involved people in an
industrial environment generating work-relevant
ideas. This trend is consistent with the findings of
the prior research by Paulus and Yang (2000) in
which group brainwriting increased the number
of ideas by a similar amount (41%). This finding
was obtained using the same method of writing
ideas in both conditions, avoiding the confound-
ing factor noted by Goldenberg et al. (2013). So
this finding suggests that even with the same
method, groups can potentially outperform indi-
viduals in brainwriting. In any case, the group
condition was not worse than the alone condition,
as is typically the case in face-to-face brainstorm-
ing. Goldenberg et al. also found that the brain-
writing procedure allowed groups to function as
well as nominal groups. Thus, brainwriting can
be recommended as an effective procedure for
groups in work settings to generate ideas. This
study is the first to demonstrate this utility in a
work environment.
The lack of effects for novelty and utility are
consistent with prior research. Comparisons of
group and individual brainstorming typically yield
Asynchronous BrAinstorming
differences in quantity but not in quality of ideas.
This finding is somewhat surprising since one
might expect interaction to be a basis for enhanc-
ing the quality of ideas. However, the group-to-
alone condition yielded more good ideas (high in
both novelty and utility). This result is consistent
with findings of other studies that an increase in
the number of ideas is typically associated with an
increase in the number of good ideas (Diehl &
Stroebe, 1987; Paulus et al., 2011; Reinig, Briggs,
& Nunamaker, 2007). Interestingly, this effect
seemed to be in large part due to the fact that num-
ber of good ideas declined in the second session of
the alone-to-group condition but not in the group-
to-alone condition. Apparently, the participants in
the group-to-alone condition were able to continue
building on the shared ideas in a fruitful session of
solitary brainstorming. Possibly, the participants
in the alone-to-group condition found the group
session somewhat constraining relative to their
prior solitary session, which may have reduced
their motivation in the group session. Alterna-
tively, the fact that the participants in this condi-
tion were allowed to share ideas from the alone
session may have reduced their motivation to gen-
erate new ideas.
However, in past studies, we have found that
providing participants extra instructions that
include encouragement to go back to previously
generated categories of ideas if they cannot think
of new ones at some point in time enhanced the
number of ideas generated (Paulus, Nakui, Put-
man, & Brown, 2006; Putman & Paulus, 2009).
Furthermore, the sharing of ideas from the alone
session could potentially increase the number of
mutually shared ideas and stimulation of new
ideas in the group. However, given the decline in
performance in the group phase of the alone-to-
group condition, it is clear that allowing reuse of
past ideas did not enhance the stimulation in that
From past studies it appears that increasing
the average quality of ideas requires that groups
be highly engaged in the idea generation pro-
cess, carefully process the shared ideas, and then
build on the ideas of others (Baruah & Paulus,
2009; De Dreu, Nijstad, Bechtoldt, & Baas,
2011; Kohn et al., 2011). This result may be dif-
ficult to demonstrate in the short sessions used
in most brainstorming studies. Although it would
have been ideal to use longer sessions in this
study, it was evident to the research personnel
involved that the participants did not have the
patience or motivation for long sessions, espe-
cially given the fact that they were participating
in two studies in sequence. Previous studies have
shown that the pace of idea generation declines
quickly over time (cf. Paulus & Dzindolet, 1993).
When the pace slows, participants presume they
are running out of ideas and use this cue as a
basis for disengaging from the task or stopping
the idea generation process (Nijstad, Stroebe, &
Lodewijkx, 1999). To keep the brainstorming
process going over an expanded period appears
to require brief breaks (Paulus et al., 2006).
These breaks may allow for cognitive rest, time
for reflection, and the overcoming of fixation
(Smith, 2003).
TABLE 2: Questionnaire Data From Study 1: Means and Standard Deviations
Item Alone to Group Group to Alone
How competitive did you feel? 2.463 (0.564) 3.130 (1.633)
How much effort did you expend? 3.167 (0.717) 3.611 (0.590)
How motivated were you? 2.926 (0.778) 3.648 (0.556)
How carefully did you read others’ ideas? 3.630 (0.423) 4.056 (0.333)
Seeing others’ ideas helped you generate more ideas 2.870 (0.498) 3.444 (0.441)
Seeing others’ ideas helped generate better ideas 2.815 (0.848) 3.444 (0.471)
How would you rate the quality of your own ideas? 2.982 (0.757) 3.648 (0.242)
How many ideas do you think you generated? 3.000 (0.479) 3.389 (0.354)
Note. Standard deviations are shown in parentheses. There were 10 groups in both the conditions. Data from one
group in each condition were missing.
10 Month XXXX - Human Factors
The group-to-alone sequence led to 63%
more ideas per person than did the alone-to-
group one. Again, this is an impressive finding
from a practical perspective. The questionnaire
items also indicate that participants had more
positive reactions to the group-to-alone sequence
than to the alone-to-group one. If these items
reflect actual processes rather than just general
positive affective reactions, they suggest that the
group-to-alone condition may increase the moti-
vation to perform the task well (increased moti-
vation and attention). Participants typically
enjoy group brainstorming more than individual
brainstorming (e.g., Paulus et al., 1995; Paulus,
Dzindolet, Poletes, & Camacho, 1993). The pos-
itive feelings generated and the related positive
effect on group processes in the first session
may carry over into the second solitary ideation
Although this study demonstrated a benefit of
the group-to-alone sequence, the degree of this
benefit may depend on a range of factors, such as
type of task (verbal, writing, electronic), length of
sessions, and level of training. For example, in
highly trained groups using verbal brainstorming,
the alone-to-group sequence was found to yield
more ideas than the group-to-alone one (Baruah &
Paulus, 2008). The training enhanced the idea
generation of both individual and group brain-
stormers. Thus, the high level of performance of
the individual brainstormers in the first session
carried over into the second group brainstorming
session. This type of pacing or entrainment effect
has been observed in other group settings (Kelly
& McGrath, 1985). Therefore, highly trained indi-
viduals discover they can generate many ideas
individually and can build on these ideas in a sub-
sequent group session. The brainwriting paradigm
may be more optimal for the group-to-alone
sequence. Since groups can generate more ideas
than individuals, there may be more ideas on
which individuals can build in a subsequent indi-
vidual brainstorming session. However, longer
sessions may be more beneficial for the alone-to-
group sequence. In long sessions, groups and indi-
viduals may begin to “run out” of ideas. In the
group-to-alone sequence, individuals may not be
able to come up with many more ideas, but in an
alone-to-group session, the group sharing may
stimulate consideration of additional categories of
ideas that may not have been considered by indi-
viduals brainstorming alone.
The ideas selected as the best ones were not
more novel or useful than the average novelty or
utility of all of the ideas the group had gener-
ated. This finding is consistent with past research
indicating that groups have a difficult time pick-
ing the best ideas from the pool they have gener-
ated (Putman & Paulus, 2009; Rietzschel et al.,
Although a particular sequence of alone/
group brainstorming might be beneficial in
specific circumstances, a simple alternation of
ideas exchange and reflection on those ideas
is most likely ideal and representative of most
real-world situations. That is, individuals may
reflect on a particular issue before a group
sharing session. After the group sharing, some
additional reflection on the shared ideas may
stimulate more ideas. These ideas in turn can
be shared with the group, leading to potential
stimulation of more novel ideas or combinations
of ideas (Kohn et al., 2011). This type of asyn-
chronous brainstorming would take advantage
of the stimulating effects of group interaction
and the benefits of solitary reflection on those
The cognitive models of group ideation sug-
gest a benefit of both group sharing and the ben-
efits of individual idea sessions after group shar-
ing (Nijstad & Stroebe, 2006; Paulus & Brown,
2003). For example, Paulus and Yang (2000)
found that group brainwriting enhanced ideation
but also that this benefit carried over into an
individual ideation session. That is, group brain-
writers generated almost as many ideas during a
subsequent individual brainwriting session as
during the group session (only a 5% decline).
Those who wrote ideas alone subsequently
showed a more typical decline in number of
ideas generated during their second solitary
writing session (30%). Madsen and Finger
(1978) found that providing solitary brainwriters
with ideas generated by three other brainstorm-
ers at the halfway point of the session increased
the number of ideas by 20% relative to those
who did not get such feedback. Thus, the expo-
sure to the ideas of others can lead to stimulation
Asynchronous BrAinstorming
of additional ideas by the associations generated
and the additional categories of ideas suggested.
Individual ideation sessions after such exposure
sessions can allow for individuals to effectively
build on these associations and category cues
without the demands of a collaborative sharing
process in which participants have to read ideas
of others while they are also trying to generate
their own ideas.
We are not aware of any published study that
has addressed the benefits of alternation in group
and individual brainstorming sessions. In Study
2, participants worked in the same groups as in
Study 1. In the group condition, participants
generated ideas on pieces of paper and shared
them with the other group members for 30 min.
The instructions and procedure were the same as
in the group brainwriting conditions of Study 1,
with the exception that the participants brain-
stormed as a group only for the 30-min period.
In the asynchronous condition, participants
alternated between writing ideas individually
and brief review sessions in which they could
read the slips of the other group members. It was
predicted that the asynchronous condition would
lead to better performance than the regular group
brainwriting condition in terms of number of
ideas and number of good ideas. The asynchro-
nous condition combines the benefits of indi-
vidual ideation with that of shared ideas. Indi-
viduals can express ideas as they occur in the
individual ideation sessions. There is no produc-
tion blocking of the exchange process. However,
the brief review sessions allow them to quickly
scan the ideas of the other group members and
receive the potential benefit of the additional
stimulation provided by such a session. Although
no published study has employed such a para-
digm, a laboratory study examining this proce-
dure found that this type of asynchronous condi-
tion was indeed beneficial (Korde, 2011).
Hypothesis 1: Asynchronous brainwriting
will lead to generation of more ideas and
more good ideas than group brainwriting.
[AQ: 7]
The same 57 participants from Study 1 worked
in groups of two or three, with 10 groups in the
two conditions. There was one group of two in
the group condition and two such groups in the
asynchronous condition. In the group brainwrit-
ing condition, participants shared ideas on slips
of paper. They were asked to use different-color
slips in each of three 10-min periods so we could
track the performance over time. They were given
the standard brainstorming instructions and were
asked to generate ideas about market and product
innovation for their company. The specific topic
provided was as follows:
We would like you to generate ideas about
market and product innovation. Any ideas
on these issues are encouraged. This can
include defining the differentiation for the
next winning mobile device, future direc-
tions, driving an idea in the organization
(influence on management, strategic plan-
ning, etc.), and new markets. These are
just some of the areas you can consider.
In the asynchronous condition, participants
alternated between 8-min individual writing ses-
sions and 3-min review sessions. We had found
this amount of review time to be sufficient in
prior unpublished laboratory studies. The
sequence for this condition was as follows: an
8-min writing session, a 3-min review session, 8
min writing, 3 min reviewing, and 8 min writ-
ing. In this condition, participants wrote for 24
min and reviewed ideas for 6 min.
The participants were instructed that they
would be alternating individual writing sessions
with sharing sessions, using different-color slips
for each idea-sharing session. During individual
writing sessions, participants were instructed to
write one idea on each slip and keep the slips in
a pile in front of them. During the review ses-
sion, they were instructed to pass their piles
among the group members and quickly skim
over the ideas. The experimenter told the groups
to stop and start each of the sessions at the appro-
priate times. After the idea generation session,
the groups were provided 15 to 20 min to go
over all the ideas generated and pick the best
three ideas. They were then given a brief ques-
tionnaire on their perceptions of the brainstorm-
ing session and their level of motivation for the
task, similar to Study 1.
12 Month XXXX - Human Factors
Average quantity (ideas per person). Consis-
tent with the predictions, the asynchronous con-
dition led to 36% more ideas than the group
condition, although this effect was not signifi-
cant, F(1, 18) = 2.933, p = .104, ηp
2 = .140 (see
Table 3 for descriptives). There was a significant
effect of time, with more ideas being generated
in the initial phases as compared to the latter
phases, F(2, 17) = 5.473, p = .015, ηp
2 = .392,
but the interaction effect was not significant,
F(2, 17) = .328, p = .725, ηp
2 = .037.
Rate of ideas per person per minute. Since the
groups had different total times to generate ideas
and there was an additional dyad in the asynchro-
nous condition, a rate of idea generation measure
may be a more appropriate measure of the relative
benefits of the two conditions. An analysis of the
rate of ideas generated per person per minute dur-
ing the idea generation sessions (not including
review time in the asynchronous condition)
showed that the rate of idea generation was sig-
nificantly higher in asynchronous condition, F(1,
18) = 8.092, p = .011, ηp
2 = .310, a 71% advantage.
As with average ideas per group, there was a sig-
nificant effect of time, F(2, 17) = 5.263, p = .017,
2 = .382, but the interaction effect was not sig-
nificant, F(2, 17) = .425, p = .661, ηp
2 = .048.
Utility and novelty. The ideas were coded for
utility and novelty using the same procedure
used in Study 1. The ICC single-measures value
was .403 and Cronbach’s alpha was .603 for util-
ity. For originality, the ICC single-measures
value was .393 and Cronbach’s alpha was .565.
As in Study 1, there were no effects for average
utility, F(1, 18) = .263, p = .614, ηp
2 = .014, or
novelty, F(1, 18) = .403, p = .533, ηp
2 = .022. A
mixed ANOVA was used to test differences in
novelty over time as well as across conditions.
However, there were no significant differences
in novelty over time, F(2, 17) = .353, p = .707,
2 = .040, and there was no significant interac-
tion effect, F(2, 17) = .853, p = .444, ηp
2 = .091.
Number of good ideas. Similar to Study 1,
ideas with scores of 3 or more on both original-
ity and utility were considered as good ideas
since that captured a reasonable number of ideas.
There were no significant differences in the
number of good ideas across conditions, F(1,
18) = 2.248, p = .151, ηp
2 = .111, or phases, F(2,
17) = 2.962, p = .079, ηp
2 = .258, and there was
no significant interaction effect, F(2, 17) = .198,
p = .822, ηp
2 = .023. However, the trend for con-
dition mirrored the results for the number of
ideas, with 35% more good ideas in the asyn-
chronous condition.
Best ideas. There were no significant differ-
ences between the conditions in rated novelty,
t(18) = .049, p = .961, and utility, t(18) = −.165,
p = .870, of the ideas generated for the best ideas
selected. The analysis was also performed
for each condition separately and revealed no
TABLE 3: Study 2: Quantity, Novelty, Utility of Brainstorming Ideas by Condition and Phase
Condition Average Quantity Rate of Ideas Novelty Utility Number of Good Ideas
Phase 1 4.667 (2.176) .583 (.271) 2.528 (0.375) 3.155 (0.199) 5.90 (3.071)
Phase 2 3.950 (1.644) .494 (.206) 2.595 (0.632) 3.186 (0.488) 4.40 (3.098)
Phase 3 3.433 (1.421) .429 (.178) 2.686 (0.605) 2.946 (0.495) 4.60 (3.134)
Best ideas 2.500 (0.774) 3.167 (0.324)
Overall 4.017 (1.582) .502 (.198) 2.603 (0.306) 3.096 (0.231) 14.90 (6.154)
Phase 1 3.517 (1.500) .352 (.150) 2.556 (0.334) 2.991 (0.412) 4.60 (2.366)
Phase 2 2.700 (1.319) .270 (.132) 2.652 (0.392) 3.108 (0.328) 3.60 (2.066)
Phase 3 2.617 (1.310) .262 (.131) 2.346 (0.443) 3.013 (0.486) 2.80 (2.201)
Best ideas 2.483 (0.747) 3.200 (0.549)
Overall 2.944 (1.190) .294 (.119) 2.518 (0.293) 3.037 (0.277) 11.00 (5.457)
Note. Standard deviations are shown in parentheses. There were 10 groups in both the conditions.
Asynchronous BrAinstorming
significant difference in novelty ratings: asyn-
chronous, t(9) = .551, p = .595; group, t(9) =
.132, p = .898. There was also no significant dif-
ference in utility ratings: asynchronous, t(9) =
−.709, p = .496; group, t(9) = −1.131, p = .287.
Questionnaire items. The questionnaire items
used in this study were similar to those used in
Study 1, and t tests were used to analyze the data
from the questionnaire items. There were no sig-
nificant differences between conditions on any
of the questionnaire items.
The asynchronous condition yielded more
ideas than did the group condition. This trend
was statistically nonsignificant for average
number of ideas but reached conventional sig-
nificance when we examined rate of ideas per
person per minute (to equate for brainstorm-
ing time). Again, these results are practically
important since this procedure led to an increase
in ideas of 36% to 71%, depending on the mea-
sure. These findings are consistent with those
from a laboratory study by Korde (2011) and
suggest that some type of alternation sequence
between group and individual brainstorming
can be beneficial for increasing the generation
of ideas. The stronger effect with the rate mea-
sure could reflect in part the fact that this mea-
sure does not equate for reading time. Reading
time was eliminated in the asynchronous condi-
tion but could not be eliminated in the group
condition since we could not track reading time
in that condition. So the rate measure included
both ideation and reading time for the group
condition. However, it is a meaningful measure
of productivity in that it indicates that asynchro-
nous participants were relatively more efficient
during their ideation time. They generated .50
ideas per minute compared to .29 for the group
condition. Even if the rate of the participants in
the asynchronous condition is determined using
the 30 min of the total session, their rate is still
higher than in the group condition (.40). It is
useful to know that the periodic review proce-
dure did not slow down the ideation process
because of the required interruptions of the idea
generation process.
Since this is the first study to systematically
address the benefit of asynchronous brainstorming
in a work environment, there is much to learn
about the optimal sequence and length factors.
There needs to be an appropriate balance between
the length of the generation and review sessions.
The sessions need to be long enough to generate
sufficient ideas for a review session. A set of long
sessions may not be as effective as a set of shorter
ones, because participants tend to reduce their idea
rate rather quickly. Shorter sessions that provide
more breaks may be helpful. Also, with shorter
sessions, participants will not be overloaded with
ideas in the review sessions and may be able to do
a more careful review.
In real-world contexts, group and alone ses-
sions may be separated by longer periods than
assessed in a laboratory context—such as hours
or days. An advantage of longer periods between
sessions is that they may provide more time for
participants to reflect on shared ideas and build
on them. However, this advantage may not be
realized unless there is some degree of structure,
since the intervening period is likely to be filled
with other tasks and activities. Moreover, there
is likely to be a significant decay in any novel
associations generated during a sharing session.
From the perspective of cognitive models of col-
laborative ideation (Nijstad & Stroebe, 2006;
Paulus & Brown, 2007), it may be important to
tap associations generated by idea sharing in
subsequent alone sessions that follow shortly
after the sharing session so that the decay of
associations or spreading activation will be lim-
ited. From that vantage point, it is likely that
many of the ideas that might be stimulated by
meetings may be lost, since we are not aware of
organizations using a practice in which the stim-
ulation of shared ideas is harvested during a sub-
sequent structured reflection session.
Although we found that the condition of
alternating review and solitary idea generation
sessions was more productive than the one that
involved continual sharing of ideas during the
brainstorming session, it is possible that the brief
breaks, which the review process provided, may
also have contributed to the effect. Previous
research has shown that brief, 3-min breaks can
enhance the performance of individual brain-
writers (Paulus et al., 2006). Brief breaks can
help overcome fixation on certain ideas or types
of ideas (Smith, 2003) or may provide beneficial
14 Month XXXX - Human Factors
cognitive rest. In idea-sharing situations, ideas
from others can stimulate new ideas (Dugosh,
Paulus, Roland, & Yang, 2000) and can also lead
to a tendency to generate similar ideas (collab-
orative fixation; Kohn & Smith, 2011). Addi-
tional studies in work settings will be required to
establish the relative impact of idea review and
sharing sessions relative to private reflection
sessions. However, it is clear from the present
study that in practice, using review sessions to
alternate with private idea generation sessions
can be a beneficial alternative to using only
group sharing. The fact that there were no differ-
ences in novelty among the conditions suggests
that the conditions did not differentially influ-
ence the extent to which participants generated
ideas similar to those of others.
In contrast to Study 1, there were no differ-
ences in participant perceptions in the two con-
ditions. Consistent with Study 1, there were no
effects for originality and novelty, and the best
ideas selected did not differ in originality and
utility from the average of all of the ideas.
Although there were no significant effects for
number of good ideas as in Study 1, the number
of good ideas reflected the pattern for number of
ideas, with more good ideas generated in the
asynchronous condition. The rater reliabilities
for the coding of novelty and originality were
lower in this study than in Study 1. The technical
problem was obviously more difficult to code
along these dimensions than the team problem
because the raters had less experience with this
type of issue and thus had fewer reference
points. Others have noted similar issues with
coding reliability for these types of measures on
technical problems (e.g., Linsey et al., 2011).
Across these two studies, we have explored
the utility of brainwriting for idea generation
in a work setting. Both studies demonstrate the
benefits of exchanging ideas by brainwriting.
This paper thus suggests that brainwriting may
be an effective approach in industrial or organi-
zational settings when there is interest in gener-
ating a large number of new ideas. We did not
do any training on the brainwriting procedure,
given our time constraints. However, it is likely
that some degree of training and experience
will greatly enhance the utility of brainwriting
(Baruah & Paulus, 2008). Brainwriting does
not appear to be a commonly used procedure in
organizational contexts, and in our experience,
participants find it somewhat awkward initially.
Most participants would probably prefer verbal
brainstorming even though it is less effective
(Paulus et al., 1993, 1995).
It is also clear from our studies that the
sequence of group and solitary ideation is impor-
tant. Study 1 provided some evidence that the
group-to-alone sequence can be more effective
for the generation of ideas than the alone-to-
group sequence. Study 2 suggests that inter-
spersing individual review sessions with group
sharing sessions may be a beneficial procedure
for increasing ideas. Therefore, one cannot sim-
ply say that it is best to generate ideas alone or
best to do it in groups. From a practical and the-
oretical standpoint, it appears that some degree
of alternation between group and individual ide-
ation will be optimal as well as natural. How-
ever, since these two studies are only part of a
small set on the role of this type of alternation, it
will be beneficial to have some more laboratory
and field studies to investigate the generality of
our findings and how these effects vary with
such factors as the type of brainstorming (ver-
bal, writing, electronic), the length of sessions,
and the degree of training.
Some research suggests that the particular
technique of brainwriting may affect the number
and quality of ideas (Linsey et al., 2011). It
would also be of interest to examine the impact
of enhancing the exchange process by allowing
for relational information processing in which
group members discuss the process and their
approach (Carmeli et al., in press). Similarly, it
would be beneficial to determine systematically
the effect of different types of instructions, such
as those related to the ability to use ideas gener-
ated in a previous session. There are no studies
that have examined group size in brainwriting
groups. There is some evidence that increased
group size can enhance number of ideas gener-
ated in electronic brainstorming (DeRosa et al.,
2007). Increasing the size of brainwriting groups
would increase the number of ideas to which
participants are exposed, an important factor in
Asynchronous BrAinstorming
stimulating additional ideas (Paulus et al., 2013).
However, the increased demands of processing
all of the slips in a large group may take away
time from individual reflection and ideation.
Thus researchers will have to examine the right
balance of time of exposure to ideas and reflec-
tion time.
We did not find any effects on the average
quality of ideas (novelty and utility). Authors of
studies of group and individual brainstorming
have typically not found differences in quality.
The most common finding is that increases in
number of ideas are associated with an increase
in the number of good ideas (novel and useful)
but not the average quality of ideas (Diehl &
Stroebe, 1987; Paulus et al., 2011). We found
results consistent with that pattern in both exper-
iments. This finding may be a natural outcome
because of the typical brainstorming instructions
that emphasize quantity and deferment of evalu-
ation. There is not an emphasis on quality in the
instructions, and Osborn (1957) assumed that
generating a lot of ideas would also result in
more good ideas. When participants are asked to
focus on generating high-quality ideas, it can
actually inhibit the generation of good ideas
since this focus slows down the idea generation
process (Paulus et al., 2011). Reinig et al. (2007)
concluded that number of good ideas is the best
measure of quality since high average quality
could be obtained by those who might generate
only a few ideas. From a practical standpoint,
having a lot of good ideas is beneficial because
they can be evaluated and built upon in subse-
quent sessions. Interestingly, Kohn et al. (2011)
found that group discussion of ideas previously
generated can actually lead to the generation of
additional high-quality ideas.
Our perspective is that it is not necessary to
generate ideas of high average quality initially.
With a good process, there is going to be good
number of high-quality ideas that can provide the
basis of high-quality innovations as creative and
developmental processes continue within an orga-
nization. It would certainly be desirable to have a
higher average quality in group exchange pro-
cesses. However, in most cases, groups or organi-
zations can focus only on a small number of inno-
vations at one time. So the key issue is the selec-
tion and development of the best ideas. We have
noted that the process of selecting the best ideas
seems to be difficult for groups and individuals
(Putman & Paulus, 2009; Rietzschel, Nijstad, &
Stroebe, 2010). There also appears to be a bias
toward more feasible ideas and some aversion to
novel ideas (Mueller, Melwani, & Goncalo, 2012;
Putman & Paulus, 2009; Rietzschel et al., 2006). It
may be best to train some people to become effec-
tive at idea evaluation and use these people to
develop a list of the top ideas for further consider-
ation. Obviously more research needs to be done
in organizations in which the entire cycle of idea
generation, selection, and innovation develop-
ment is examined. The present study is the first in
which both a variety of ways to increase idea gen-
eration and the ideal selection process were exam-
ined. Hopefully, authors of future studies will have
the opportunity to build on this type or research
and follow the impact on the actual development
of innovative directions.
It will also be important to learn more about
ways to enhance the quality of ideas generated
in groups. There is some evidence that increased
diversity or increased training can increase the
quality of ideas (Baruah & Paulus, 2008; Nakui,
Paulus, & van der Zee, 2011). Motivating par-
ticipants to process shared ideas more deeply or
to try combining shared ideas may be helpful
in increasing the quality of the ideas (De Dreu
et al., 2011; Kohn et al., 2011).
An extensive literature has been developed
from controlled laboratory and some field stud-
ies that suggests ways to improve the innovative
potential of teams. Studies that involve evaluat-
ing the relevance of these findings to the actual
performance of work teams will be of great
value to enhancing the innovative output of
organizations. It will also be important for
authors of future research in work settings to
examine the role of diversity in expertise for
innovative outcome in teams. The literature on
the impact of such diversity on team perfor-
mance is quite mixed (e.g., Bell, Villado, Luka-
sik, Belau, & Briggs, 2011), with some research-
ers finding little overall benefit (van Knippen-
berg & Schippers, 2007). However, in the
domain of creativity, it appears that both func-
tional and demographic or personal diversity
can be quite beneficial for innovation (Paulus &
Van der Zee, 2015). This result is particularly the
16 Month XXXX - Human Factors
case when the task is quite relevant to the diver-
sity and group members have a positive attitude
toward diversity (Nakui et al., 2011).
Although the participants generated a number
of good ideas, we do not know whether these ideas
were integrated in future efforts and led to actual
innovations of products or affected the way teams
worked together. The manager of this unit left
shortly after our study, so there may not have been
a systematic follow-up effort to build on the gen-
erated ideas. Such follow-up requires strong lead-
ership and persistence in evaluating the most
promising ideas and developing useful innova-
tions. The extensive work on team innovation has
provided some helpful guidelines as to the factors
that are important for this to occur (Hülsheger
et al., 2009; Paulus et al., 2011; West, 2008).
There are a number of evident limitations of
this research project. Our modest sample size
limited our statistical power. The participants
represented a large majority of the employees of
the company. All of the participants were invited
to be part of the project, so the sample represents
the total sample we were able to tap at this site.
The effects obtained and the trends are consistent
with the predictions and some prior studies. In the
case of increased percentage of ideas and good
ideas produced in some conditions, the findings
appear to have some practical value. This gives
us confidence that the theories and findings in the
brainstorming and group creativity literature are
applicable to work settings.
We included three dyads in our sample (one
in one condition and two in another). There is no
research on the effect of group size on brainwrit-
ing, but it is possible that having one more dyad
in one condition could have influenced the
results. However, analyses in which the dyads
were eliminated did not lead to any change in
conclusions or effects observed. The two studies
were run in sequence, so it is possible that the
results of the study that was done second (actu-
ally Study 1 in this paper) might have been influ-
enced by the condition that these participants
experienced in the first study. An analysis of the
influence of the prior condition on the number of
ideas generated in the second study revealed no
effects that even approached significance.
Another limitation is that the results of this
study may be affected by the particular country
in which it was conducted and the type of popu-
lation. The population consisted mostly of engi-
neers and computer scientists. There is practi-
cally no solid information or theory on the how
the effectiveness of different group ideation pro-
cedures is influenced by the type of organization
or workers. The unit we studied was a compo-
nent of a company that was rated at that time as
one of the top 10 to 20 companies in the world by
a number of prestigious surveys. Although this
may have been a special population, we expect
that the beneficial effects of the procedures
examined in this study should apply to a broad
range of populations. Our impression is that a
key factor in the success of innovation interven-
tions in organizations will depend on not only the
effectiveness of the procedures but also the moti-
vational climate. That is why much research has
noted the importance of transformational or
inspirational leadership for effective team inno-
vation (Eisenbeiss, van Knippenberg, & Boerner,
2008). There is a need to more effectively inte-
grate the literature on the idea generation pro-
cesses (group creativity literature) and that on
implementation (team innovation literature; Pau-
lus et al., 2011; Paulus & van der Zee, 2004).
The research reported in this paper was supported
by Collaborative Grant BCS 0729305 to the first
author from the National Science Foundation, which
included support from the Deputy Director of National
Intelligence for Analysis and collaborative grants from
the National Science Foundation (CreativeIT 0855825
and INSPIRE BCS 1247971). Any opinions, findings,
and conclusions or recommendations expressed in this
material are those of the authors and do not necessarily
reflect the views of the National Science Foundation.
We would like to thank the editor and three reviewers
for their helpful feedback on earlier drafts of this paper.
We also thank Eyal Nagar for his support during this
research project.
Brainwriting can be an effective technique for
idea generation in work settings.
Group brainwriting can enhance the number of ideas
generated by groups relative to nominal groups.
Asynchronous BrAinstorming
The group-to-alone sequence may be more opti-
mal for brainwriting than the alone-to-group
Periodic review of others’ ideas may be more ben-
eficial than a continual exchange of ideas in a ses-
The important issue may not be whether group or
individual brainstorming is best but what is the best
combination of solitary and collaborative ideation.
Amabile, T. M. (1996). Creativity in context. Boulder, CO: West-
view Press.
Barki, H., & Pinsonneault, A. (2001). Small group brainstorm-
ing and idea quality: Is electronic brainstorming the most
effective approach? Small Group Research, 32, 158–205.
Baruah, J., & Paulus, P. (2008). Effects of training on idea gen-
eration in groups. Small Group Research, 39, 523–541.
Baruah, J., & Paulus, P. (2009). Enhancing group creativity: The
search for synergy. In E. A. Mannix, M. A. Neale, & J. A. Gon-
calo (Eds.), Research on managing groups and teams: Vol. 12.
Creativity in groups (pp. 29–56). Bingley, UK: Emerald Group.
Bell, S. T., Villado, A. J., Lukasik, M. A., Belau, L., & Briggs, A.
(2011). Getting specific about demographic diversity variable
and team performance relationships: A meta-analysis. Jour-
nal of Management, 37, 709–743.
Brown, V. R., & Paulus, P. B. (2002). Making group brainstorming
more effective: Recommendations from an associative mem-
ory perspective. Current Directions in Psychological Science,
11, 208–212. doi:10.1111/1467-8721.00202
Carmeli, A., & Azerual, B. (2009). How relational capital and
knowledge combination capability enhance the performance of
knowledge work units in a high-technology industry. Strategic
Entrepreneurship Journal, 3, 85–103. doi:10.1002/sej.63
Carmeli, A., Dutton, J. E., & Hardin, A. E. (in press). Respect as
an engine for new ideas: Linking respectful engagement, rela-
tional information processing, and creativity among employees
and teams. Human Relations.
Carmeli, A., Gelbard, R., & Reiter-Palmon, R. (2013). Leadership,
creative problem-solving capacity, and creative performance:
The importance of knowledge sharing. Human Resource Man-
agement, 52, 95–122. doi:10.1002/hrm.21514
Chan, J., Fu, K., Schunn, C. D., Cagan, J., Wood, K., & Koto-
vsky, K. (2011). On the benefits and pitfalls of analogies for
innovative design: Ideation performance based on analogical
distance, commonness, and modality of examples. Journal of
Mechanical Design, 133, 081004-1-11. doi:10.1115/1.4004396
Chan, J., & Schunn, C. (2014). The impact of analogies on creative
concept generation: Lessons from an in vivo study in engineer-
ing design. Cognitive Science. Advance online publication.
Collins, C. J., & Smith, K. G. (2006). Knowledge exchange and
combination: The role of human resource practices in the per-
formance of high-technology firms. Academy of Management
Journal, 49, 544–560. doi:10.5465/AMJ.2006.21794671
De Dreu, C. K. W., Nijstad, B. A., Bechtoldt, M. N., & Baas, M.
(2011). Group creativity and innovation: A motivated information
processing perspective. Psychology of Aesthetics, Creativity, and
the Arts, 5, 81–89. doi:10.1037/a0017986
Dennis, A. R., & Williams, M. L. (2003). Electronic brainstorming:
Theory, research, and future directions. In P. B. Paulus & B. A.
Nijstad (Eds.), Group creativity: Innovation through collabo-
ration (pp. 160–180). New York, NY: Oxford University Press.
Dennis, A. R., & Williams, M. L. (2005). A meta-analysis of group
size effects in electronic brainstorming: More heads are better
than one. International Journal of e-Collaboration, 1, 24–42.
DeRosa, D. M., Smith, C. L., & Hantula, D. A. (2007). The medium
matters: Mining the long-promised merit of group interaction
in creative idea generation tasks in a meta-analysis of the elec-
tronic group brainstorming literature. Computers in Human
Behavior, 23, 1549–1581. doi:10.1016/j.chb.2005.07.003
Diehl, M., & Stroebe, W. (1987). Productivity loss in brain-
storming groups: Toward the solution of riddle. Jour-
nal of Personality and Social Psychology, 53, 497–509.
Dornburg, C. C., Stevens, S. M., Hendrickson, S. M. L., &
Davidson, G. S. (2009). Improving extreme-scale prob-
lem solving: Assessing electronic brainstorming effective-
ness in an industrial setting. Human Factors, 51, 519–527.
Dugosh, K. L., Paulus, P. B., Roland, E. J., & Yang, H. (2000). Cog-
nitive stimulation in brainstorming. Journal of Personality and
Social Psychology, 79, 722–735. doi:10.1037/0022-3514.79.5.722
Dunbar, K. N. (1997). How scientists think: On-line creativity and
conceptual change in science. In T. B. Ward, S. M. Smith, &
J. Vaid (Eds.), Creative thought: An investigation of concep-
tual structures and processes (pp. 461–493). Washington, DC:
American Psychological Association.
Dunnette, M. D., Campbell, J., & Jaastad, K. (1963). The effect
of group participation on brainstorming effectiveness for 2
industrial samples. Journal of Applied Psychology, 47, 30–37.
Eisenbeiss, S. A., van Knippenberg, D., & Boerner, S. (2008).
Transformational leadership and team innovation: Integrating
team climate principles. Journal of Applied Psychology, 93,
Goldenberg, O., Larson, J. R., & Wiley, J. (2013). Goal instruc-
tions, response format, and idea generation in groups. Small
Group Research, 44, 227–256. doi:0.1177/1046496413486701
Hargadon, A. B., & Bechky, B. A. (2006). When collections of
creatives become creative collectives: A field study of prob-
lem solving at work. Organization Science, 17, 484–500.
Heslin, P. A. (2009). Better than brainstorming? Potential boundary
conditions to brainwriting for idea generation in organizations.
Journal of Occupational and Organizational Psychology, 82,
129–145. doi:0.1348/096317908X285642
Hülsheger, U. R., Anderson, N., & Salgado, J. F. (2009). Team-
level predictors of innovation at work: A comprehensive meta-
analysis spanning three decades of research. Journal of Applied
Psychology, 94, 1128–1145. doi:10.1037/a0015978
Kelly, J. R., & McGrath, J. E. (1985). Effects of time limits and
task types on task performance and interaction of four-person
groups. Journal of Personality and Social Psychology, 49,
395–407. doi:10.1037/0022-3514.49.2.395
Kohn, N. W., Paulus, P. B., & Choi, Y. (2011). Building on the
ideas of others: An examination of the idea combination pro-
cess. Journal of Experimental Social Psychology, 47, 554–561.
18 Month XXXX - Human Factors
Kohn, N. W., & Smith, S. M. (2011). Collaborative fixation:
Effects of others’ ideas on brainstorming. Applied Cognitive
Psychology, 25, 359–371. doi:10.1002/acp.1699
Korde, R. M. (2011). Asynchronous idea generation (Unpublished
master’s thesis). University of Texas at Arlington.
Larson, J. R. (2010). In search of synergy in small group perfor-
mance. New York, NY: Psychology Press.
Linsey, J. S., Clauss, E. F., Kurtoglu, T., Murphy, J. T., Wood, K.
L., & Markman, A. B. (2011). An experimental study of group
idea generation techniques: Understanding the roles of idea
representation and viewing methods. Journal of Mechanical
Design, 133(3), 031008-1-15. doi:10.1115/1.4003498
Madsen, D. B., & Finger, J. R. Jr. (1978). Comparison of a writ-
ten feedback procedure, group brainstorming, and individual
brainstorming. Journal of Applied Psychology, 63, 120–123.
Meade, M. L., Nokes, T. J., & Morrow, D. G. (2009). Expertise
promotes facilitation on a collaborative memory task. Memory,
17, 39–48. doi:10.1080/09658210802524240
Michinov, N. (2012). Is electronic brainstorming or brainwriting
the best way to improve creative performance in groups? An
overlooked comparison of two idea generation techniques.
Journal of Applied Social Psychology, 42, 222–243. doi:10.1
Michinov, N., Jamet, E., Métayer, N., & Le Hénaff, B. (2015). The
eyes of creativity: Impact of social comparison and individual
creativity on performance and attention to others’ ideas during
electronic brainstorming. Computers in Human Behavior, 42,
57–67. doi:10.1016/j.chb.2014.04.037
Mueller, J. S., Melwani, S., & Goncalo, J. A. (2012). The bias
against creativity: Why people desire but reject creative ideas.
Psychological Science, 23, 13–17. doi:10.1037/t03782-000
Mullins, D., Rummel, N., & Spada, H. (2011). Are two heads bet-
ter than one? Differential effects of collaboration on students’
computer supported learning in mathematics. International
Journal of Computer-Supported Collaborative Learning, 6,
421–443. doi:10-1007/s11412-011-9122-z
Nagasundaram, M., & Dennis, A. R. (1993). When a group is
not a group: The cognitive foundation of group idea gen-
eration. Small Group Research, 24, 463–489. doi:10.1177/
Nakui, T, Paulus, P. B., & van der Zee, K. I. (2011). The role of
attitudes in reactions to diversity in work groups. Journal of
Applied Social Psychology, 41, 2327–2351. doi:10.1111/
Nijstad, B. A., & Stroebe, W. (2006). How the group affects the
mind: A cognitive model of idea generation in groups. Person-
ality and Social Psychology Review, 10, 186–213. doi:10.1207/
Nijstad, B. A., Stroebe, W., & Lodewijkx, H. F. M. (1999). Per-
sistence of brainstorming groups: How do people know when
to stop? Journal of Experimental Social Psychology, 35, 165–
185. doi:10.1006/jesp.1998.1374
Nijstad, B. A., van Vianen, A. E. M., Stroebe, W., & Lodewijkx,
H. F. M. (2004). Persistence in brainstorming: Exploring stop
rules in same-sex groups. Group Processes & Intergroup Rela-
tions, 7, 195–206. doi:10.1177/1368430204046107
Nokes-Malach, T. J., Meade, M. L., & Morrow, D. G. (2012). The
effect of expertise on collaborative problem solving. Think-
ing & Reasoning, 18, 32–58. doi:org/10.1080/13546783
Osborn, A. F. (1957). Applied imagination: Principles and proce-
dures of creative thinking. New York, NY: Scribner.
Paulus, P. B., & Brown, V. (2003). Enhancing ideational creativ-
ity in groups: Lessons from research on brainstorming. In P.
B. Paulus & B. A. Nijstad (Eds.), Group creativity: Innovation
through collaboration (pp. 110–136). New York, NY: Oxford
University Press.
Paulus, P. B., & Brown, V. R. (2007). Toward more creative
and innovative group idea generation: A cognitive-social-
motivational perspective of group brainstorming. Social and
Personality Psychology Compass, 1, 248–265. doi:10.1111/
Paulus, P. B., & Coskun, H. (2012). Group creativity. In J. M.
Levine (Ed.), Group processes (pp. 215–239). Amsterdam,
Netherlands: Elsevier.
Paulus, P. B., Dugosh, K. L., Dzindolet, M. T., Coskun, H.,
& Putman, V. L. (2002). Social and cognitive influences
in group brainstorming: Predicting production gains and
losses. European Review of Social Psychology, 12, 299–325.
Paulus, P. B., & Dzindolet, M. T. (1993). Social influence processes
in group brainstorming. Journal of Personality and Social Psy-
chology, 64, 575–586. doi:10.1037/0022-3514.64.4.575
Paulus, P. B., Dzindolet, M. T., & Kohn, N. (2011). Collabora-
tive creativity: Group creativity and team innovation. In M.
D. Mumford (Ed.), Handbook of organizational creativity
(pp. 327–357). New York, NY: Elsevier.
Paulus, P. B., Dzindolet, M. T., Poletes, G. W., & Camacho, L.
M. (1993). Perception of performance in group brainstorming:
The illusion of group productivity. Personality and Social Psy-
chology Bulletin, 19, 78–89. doi:10.1177/0146167293191009
Paulus, P. B., Kohn, N. W., & Arditti, L. E. (2011). Effects of quan-
tity and quality instructions of brainstorming. Journal of Creative
Behavior, 45, 38–46. doi:10.1002/j.2162-6057.2011.tb01083.x
Paulus, P. B., Kohn, N. W., Arditti, L., & Korde, R. M. (2013).
Understanding the group size effect in electronic brain-
storming. Small Group Research, 44, 332–352. doi:10.1177/
Paulus, P. B., Larey, T. S., & Ortega, A. H. (1995). Performance and
perception of brainstormers in an organizational setting. Basic
and Applied Social Psychology, 17, 249–265. doi:10.1207/
Paulus, P. B., Nakui, T., Putman, V. L., & Brown, V. R. (2006).
Effects of task instructions and brief breaks on brainstorming.
Group Dynamics: Theory, Research, and Practice, 10, 206–
219. doi:10.1037/1089-2699.10.3.206
Paulus, P. B., & van der Zee, K. I. (2004). Should there be a
romance between teams and groups? Journal of Occu-
pational and Organizational Psychology, 77, 475–480.
Paulus, P. B., & van der Zee, K. I. (2015). Creative processes in
culturally diverse teams. In S. Otten, K. I. van der Zee, & M.
Brewer (Eds.), Towards inclusive organizations: Determinants
of successful diversity management at work (pp. 108–131).
London, UK: Psychology Press.
Paulus, P. B., & Yang, H. (2000). Idea generation in groups: A
basis for creativity in organizations. Organizational Behav-
ior and Human Decision Processes, 82, 76–87. doi:10.1006/
Petre, M. (2004). How expert engineering teams use disciplines
of innovation. Design Studies, 25, 477–493. doi:10.1016/
Putman, V. L., & Paulus, P. B. (2009). Brainstorming, brainstorm-
ing rules, and decision making. Journal of Creative Behavior,
43, 23–39. doi:10.1002/j.2162-6057.2009.tb01304.x
Asynchronous BrAinstorming
Reinig, B. A., Briggs, R. O., & Nunamaker, J. F. (2007). On the
measurement of ideation quality. Journal of Management
Information Systems, 23, 143–161. doi:10.2753/MIS0742-
Rietzschel, E. F., Nijstad, B. A., & Stroebe, W. (2006). Productivity
is not enough: A comparison of interactive and nominal brain-
storming groups on idea generation and selection. Journal of
Experimental Social Psychology, 42, 244–251. doi:10.1016/j.
Rietzschel, E. F., Nijstad, B. A., & Stroebe, W. (2010). The selec-
tion of creative ideas after individual idea generation: Choosing
between creativity and impact. British Journal of Psychology,
101, 47–68. doi:10.1348/000712609X414204
Smith, S. (2003). The constraining effects of initial ideas. In P. B.
Paulus & B. A. Nijstad (Eds.), Group creativity: Innovation
through collaboration (pp. 15–31). New York, NY: Oxford
University Press.
Van Gundy, A. B. (1983). Brainwriting for new product ideas: An
alternative to brainstorming. Journal of Consumer Marketing,
1, 67–74. doi:10.1108/eb008097
van Knippenberg, D., & Schippers, M. C. (2007). Work group
diversity. Annual Review of Psychology, 58, 515–541.
West, M. A. (2008). Effective teams in organizations. In C. Nik
(Ed.), An introduction to work and organizational psychology:
A European perspective (2nd ed., pp. 305–328). Malden, MA:
Paul B. Paulus is a Distinguished University Profes-
sor at the University of Texas at Arlington. He
received his PhD from the University of Iowa in
1971. His primary research interest is collaborative
creativity with a recent focus on creativity in net-
Runa M. Korde completed her PhD in experimental
psychology in 2014 at the University of Texas at
Arlington. Her research topics include group brain-
storming and the influence of the physical environ-
ment on creativity. She has an MA (University of
Mumbai, 2007) and an MS (University of Texas at
Arlington, 2011) in psychology.
Jubilee J. Dickson received her master of arts
degree in clinical psychology in 2010 from Mid-
western State University in Wichita Falls, Texas, and
is currently in her fifth year in the experimental
psychology PhD program at the University of Texas
at Arlington.
Abraham Carmeli is a faculty member at Tel Aviv
University. He received his PhD from the University
of Haifa in 2000. His current research interests
include leadership and top management teams, stra-
tegic decision-making processes, decline and fail-
ures in organizations, positive work relationships,
knowledge creation and integration, and creativity
and innovative behaviors.
Ravit Cohen-Meitar is a managing partner of a con-
sulting firm in Israel. She received her PhD from the
Graduate School of Business Administration at Bar-
Ilan University in 2007. Her research interests
include creativity, meaningfulness, self-leadership
skills, and identification processes.
Date received: April 26, 2014
Date accepted: December 27, 2014
... Substantiating, studies showed that brainstorming is one Trends Sci. 2021; 18 (22): 500 2 of 11 of the cognitive stimulating methods that promotes group creativity [15,19]. Brainstorming enhances all idea generation under the 4 basic rules: 1) to generate as many ideas as possible; 2) no one is to criticize an idea; 3) to produce wild ideas; and 4) to combine ideas or improve on each other's ideas [20]. ...
... For BWr, group members generate and share ideas written on paper or sticky notes, without talking, for 4 -5 rounds of 8 min/round [15]. This method appears to provoke a significantly higher level of originality than does CB [22]. Nonetheless, this method shows disadvantage extent to which remaining fixed on certain ideas due to this method influencing participants dig deeply into specific categories of idea generated as time progresses [15,21]. ...
... The main reason for this was that the learning process was based on the experiential learning process that encouraged participants to Trends Sci. 2021; 18 (22): 500 8 of 11 increase critical, convergent and divergent thinking [4,41,43]. These results are consistent with Kolb [43], Hasan and colleagues [44] that learning program consisted of various activities and team-learning interaction through the process of gaining new knowledge or experiences, thinking, reflection, and handson could enrich IWB. ...
Full-text available
The ability to learn and innovate has been known as a key-factor for nurses to deliver quality of care and patient safety. To be successful in nursing innovation, nurses should perform sound innovative behaviors; thus, their innovative work behavior (IWB) should be promoted. A comprehensive learning program aimed at improving IWB with effective cognitive stimulation could enhance innovation abilities and quality of innovative idea performance. This study aimed to evaluate the effectiveness of a learning program that uses Modified Hybrid Brainstorming (MHB) techniques in increasing IWB and improving the quality of ideas generated (fluency, flexibility and originality) among nurses. A sample of 60 nurses were randomly assigned to a within group pre-test and post-test study design. The learning program with the MHB method was developed and implemented for 3 sessions workshop. IWB was measured pre- and 12 weeks post-program. Pre- and post-training session fluency, flexibility, and originality of generated ideas were also measured. The percentage of absolute agreement and the index of item- objective congruence of research instruments were met. Data were analyzed using descriptive statistics, and paired t-test. The results revealed that the average IWB scores of nurses after the implementation of the MHB learning program increased significantly (p < 0.05). There was also a significant difference in post-test scores (p < 0.05) of fluency, flexibility, and originality of ideas. This study showed that offering the MHB learning program to nurses is a feasible method to improve IWB and innovative idea.
... Pour d'autres chercheurs, le problème n'est probablement pas lié à la taille du groupe, mais au processus de coordination et d'organisation de celui-ci (Linsey et al., 2005). Selon la littérature de ces dernières années, l'équilibre d'interaction et de participation interindividuelles au sein d'un groupe détermine sa performance générale tout en réduisant les conflits et en augmentant le niveau de communication et de cohésion (Paulus et al., 2015). En ce sens, l'une des recherches scientifiques réalisées ces dernières années, et citée ci-dessus, a également montré que la taille du groupe n'avait pas d'impact sur la performance collective générale d'un groupe (Duhigg, 2016). ...
... Dans cette optique, nous pouvons identifier un facteur de motivation intrinsèque en l'absence de récompenses externes : l'interaction sociale (Nadav et al., 2011). (Paulus et al., 2015), en profitant, comme nous l'avons vu dans le cadre théorique, des avantages des interactions hétérogènes sur l'inventivité d'un groupe d'individus (Dezsö et Ross, 2011 ;Loyd et al., 2013 ;Phillips et al., 2006 ;Richard, 2000). ...
... Pendant les trois mois du processus de conception, les groupes expérimentaux et de contrôle ont communiqué uniquement de manière asynchrone en utilisant la méthode de conception décrite ci-dessus. Certaines recherches ont montré les avantages de l'interaction et de la communication asynchrones sur les interactions et communications synchrones(Paulus et al., 2015), mais, dans le cas particulier de cette recherche, cette forme d'interaction et de communication a des avantages spécifiques à notre contexte, ainsi qu'à l'expérimentation. des parents d'élèves et des responsables de formation ou des entreprises publiques, entraînant potentiellement des comportements d'inhibition dans les interactions au cours du processus de conception en raison de la hiérarchie des statuts et des représentations que celle-ci implique. ...
Design groups are often expected to be creative and inventive. The performance of an inventive design group can be measured by three factors: the total number of contributions (ideas), the even distribution of the contribution rate within the group, and the exploration of new ideas. Since digital technologies challenge the uses, form, design and interactions proposed by digital learning artifacts, many digital resources for academic learning are similar to digital facsimiles of print resources. Thus, the supposed potential of digital techniques is not being fully realized and there is a lack of inventiveness in the design of digital versions of learning artifacts. This thesis proposes a design method that promotes collaborative interactions - as opposed to cooperative interactions - that would influence the level of inventiveness in the process of designing digital learning artifacts. The cooperative organization organizes shared design tasks that are distributed among educational players who are experts in their fields, whereas the collaborative organization relies on the confrontation of views, ideas and contributions of all participants in design workshops that bring together groups of educational players with heterogeneous profiles and competencies. To test our hypothesis, we chose the digital textbook as a digital learning artifact, as it is emblematic of educational resources and lends itself both to a simple digitization of printed textbooks and to transpositions that benefit from the new functionalities made possible by the use of digital technology. We proposed an experimental study by forming six cross-category design groups composed of all the actors of the educational system and working online. In this way, we traced and quantified all the online interactions between the participants in each group during the three months of the design process. We found 1) that collaborative interactions in the design process improved the quality of the flow of ideas compared to the cooperative design groups that divided the work; 2) a positive correlation between the collaboration coefficient of each design group and the level of inventiveness of the designed artifact; and 3) a correlation at the individual level between the number of confrontations with a given idea during the design process and the probability of adopting it. We observed that the designed artifact could be a representation of the collective flow of ideas of the entire group, with the group's ideas being translated into the designed artifact. We postulate that this artifact could then have the potential to influence those same designers. Following this study, we tested the proposed design method several times to make it evolve. The new versions of the design method use both collaborative and cooperative design work sessions depending on the nature of the task to be performed in each step of the design process. Finally, given that we are interested in the collaborative and cooperative dynamics in the design method proposed by this research, it is important to note the importance of taking into account the risks of simplification assumed in this thesis regarding methodological choices when interpreting the results.
... One way to interpret our findings in the larger context of team innovation is to consider one classic process suggestion for creative thinking in teams: asynchronous brainstorming (Girotra, Terwiesch, & Ulrich, 2010;Paulus, Korde, Dickson, Carmeli, & Cohen-Meitar, 2015). ...
... However, it is possible that if teams were brought together later (e.g., start as a differentiated structure and then move to an integrated structure), process loss may not have occurred to the same degree and communication may not be as detrimental to team outcomes. Similar advice for alternation between creative work in subsets of a larger entity and a larger entity is suggested in work on individual and group brainstorming effectiveness (Paulus et al., 2015). More temporally complex examinations of this phenomenon and the effects of leadership over time (i.e., Halbesleben, Novicevic, Harvey, & Buckley, 2003) would help explain the exact mechanisms behind the effects of team structure on innovative performance outcomes. ...
... Using the principles of design-based research is another strength, as design-based research studies can play an important role in the advancement of theory and practice in designing or redesigning workbased learning environments and assessment programs [14]. Although exact data on its validity and reliability are still lacking, the method of brainwriting has been presented as a novel and efficient alternative to brainstorming that can rapidly inform program implementation at minimal time and cost [36][37][38]. ...
Full-text available
Background A quality framework for hospital-based physiotherapy is lacking. This study aims to design a framework, building on the currently available literature, to improve the quality of hospital-based physiotherapy. Methods A multidisciplinary panel of six representatives of hospital-based physiotherapy and their key stakeholders (patients, medical specialists, hospital management and professional association) was set up. We used brainwriting to sample ideas and the ‘decision-matrix’ to select the best ideas. Results The first round of brainwriting with an online panel of six experienced participants yielded consensus on seven possible methods for quality improvement of hospital-based physiotherapy [1]: continuing education [2] ,feedback on patient reported experience measures and patient reported outcome measures [3] ,a quality portfolio [4] ,peer observation and feedback [5] ,360 degree feedback [6] ,a management information system, and [7] intervision with intercollegiate evaluation. Placing these methods in a decision matrix against four criteria (measurability, acceptability, impact, accessibility) resulted in a slight preference for a management information system, with almost equal preference for five other methods immediately thereafter. The least preference was given to a 360-degree feedback. Conclusions In the design of a framework for improving the quality of hospital-based physiotherapy, all seven suggested methods were perceived as relevant but differed in terms of advantages and disadvantages. This suggests that, within the framework, a mixture of these methods may be desirable to even out respective advantages and disadvantages.
... Brainwriting [16][17][18][19] is an effective technique for exposing participants to previously generated ideas to stimulate their thinking. Each participant writes three ideas on a worksheet and passes it to the next participant. ...
Full-text available
Brainstorming is an effective technique for offline ideation although the number of participants able to join an ideation session and suggest ideas is limited. To increase the diversity and quality of the ideas suggested, many participants with various backgrounds should be able to join the session. We have devised an evolutionary computation-assisted brainwriting method for large-scale online ideation. In this method, participants not only suggest ideas but also evaluate ideas previously suggested by other participants. The evaluation results are used in the evolutionary computation to identify good ideas to which the participants can be exposed via a brainwriting-like interface. We compared the performance of the proposed method with that of a simple online brainwriting method for large-scale online ideation with more than 30 participants. The proposed method enhanced robustness of idea quality improvement due to preferentially exposing the participants to good ideas.
The chapter investigates the role of brainstorming as an educational tool on the basis of the author's experience in transnational education, and a review of the literature. Brainstorming is an ideation technique invented by Alex Osborn in the 40s. Still popular today, it has found numerous applications in education. Despite its diffusion, a large body of literature dismisses brainstorming as less effective in comparison to individuals working alone or other techniques in terms of quantity and quality of idea produced. Advocates of brainstorming claim that when done properly it can be very powerful in generating breakthrough ideas, and studies show that factors such as the presence of a facilitator, training, and the right time and type of problem assigned can greatly change the outcome of a session. After reviewing this contrasting literature, the author concludes that these studies ignore the cultural origin of brainstorming and attention should be paid to the cultural implications of the technique in transnational contexts outside the US.
Generating creative ideas in groups is a common process in many contexts. However, research has shown that the outcomes of group idea generation are often lower than those obtained without collaboration. The effectiveness of group idea generation is influenced by the method of sharing ideas, the task structure, and the personal characteristics of the group members. The use of electronic or writing modalities for idea exchange, appropriate instructions, brief breaks, challenging goals, and focusing on one issue at a time can enhance group performance. Task-relevant diversity in group composition and some degree of training in effective group processes also enhance the performance of idea generation groups. Alternation of group and solitary idea generation sessions can also be beneficial.
Objective The aim of this study was to examine the potential benefits of multidisciplinarity among agri-food researchers working in small groups to generate ideas to stimulate innovation in the context of a laboratory project. Background Research on the role of multidisciplinarity in scientific research teams remains limited, particularly regarding the generation of ideas to innovate in a real laboratory project, and on a task with a real challenge for innovation. Method Researchers and agri-food research staff were assigned to small groups of either multidisciplinary or unidisciplinary composition to produce ideas on a cross-cutting theme for an innovative laboratory project using an electronic “brainwriting” application. Results A greater depth in idea generation (number of ideas per category) was observed in the multidisciplinary condition than in the unidisciplinary condition. Conclusion The main benefits of this study were to experimentally examine the effects of multidisciplinarity in small scientific research groups on the production of ideas in a field study conducted on the premises of an agri-food laboratory. Application This study provides advice on how to promote innovative projects by stimulating ideation processes, which includes constructing small multidisciplinary groups and using an electronic “brainwriting” technique.
Full-text available
: Changes in the 21st century have apparently led to the transformation of the structure of education and training activities and required a different perspective on the education and training process. At this point, education programs emphasize the ability of individuals to evaluate real-time problems in a multifaceted and creative way, to work in harmony and to use their high-level thinking skills. In order to develop creativity skills in individuals and to produce, evaluate and apply innovative solutions, social processes should be given more importance rather than individual efforts. The most important step to create a creative education ecosystem in line with changing conditions is to place collaborative creativity into the center of education. For this reason, it is important to reveal how collaborative creativity is conceptualized, especially in educational environments. The objective of this research is, in this context, to reveal collaborative creativity and its theoretical basis, structure, components, models and applications in the context of education at the theoretical level. In the research, a compilation was made on collaborative creativity with a focus on education and scientific studies related to the subject were examined by means of the literature review method. Thus, information about collaborative creativity, which is an innovative learning model in education, and its reflections on education was collected and the subject was addressed from different perspectives. It can be said that the arguments of this study are very important in terms of evaluating the role of collaborative creativity in education as well as being explanatory about its basic elements and components.
Full-text available
The creation of novel ideas often takes place in social contexts. Past research on brainstorming and other types of group tasks have shown that groups often perform poorly in comparison to conditions in which tasks are done individually. Past reviews have evaluated various factors that may be responsible for the production losses in group brainstorming (Diehl and Stroebe, 1987; Mullen, Johnson, and Salas, 1991; Stroebe and Diehl, 1994). We re-evaluate these factors in light of subsequent research in our laboratory and demonstrate conditions under which groups can perform quite well in comparison to similar groups of individuals (nominal groups). We present a comprehensive model for predicting production gains and losses. Research on cognitive and social factors that may enhance brainstorming in groups is summarized.
Full-text available
It is apparent that a broad range of factors can increase innovation in teams. It appears that with the right people, the right supporting, motivational and task contexts, and effective social and cognitive processes, teams can be highly innovative. This outcome is probably not surprising and to some extent states the obvious. Such excellence may not be inevitable in teams, since this configuration adds a level of complexity. Team members have to coordinate effectively, efficiently and adequately share and combine their relevant knowledge, select the best ideas, and effectively implement them. It may take considerable training and experience for teams to excel at team creativity. There is a need for studies that provide objective measures of processes and outcomes in real world settings with appropriate comparison groups. Laboratory studies of creativity in short-term groups suggest that groups will often under perform and will exceed non-interactive baselines only under conditions that are optimal for group idea exchange. However, the fact that laboratory studies have been able to find synergy with ad-hoc groups in short-term settings suggests that teams which involve members suited for teamwork, who have worked together for some time and have the diverse perspectives needed for a problem, should also be able to achieve synergistic outcomes under the right conditions.
Full-text available
Research on group brainstorming has demonstrated that it is less effective for generating large numbers of ideas than individual brainstorming, yet various scholars have presumed that group idea sharing should enhance cognitive stimulation and idea production. Three experiments examined the potential of cognitive stimulation in brainstorming. Experiments 1 and 2 used a paradigm in which individuals were exposed to ideas on audiotape as they were brainstorming, and Experiment 3 used the electronic brainstorming paradigm. Evidence was obtained for enhanced idea generation both during and after idea exposure. The attentional set of the participant and the content of the exposure manipulation (number of ideas, presence of irrelevant information) influenced this effect. These results are consistent with a cognitive perspective on group brainstorming.
Full-text available
Electronic brainstorming (EBS) has been a focus of academic research since the 1980s. The results suggest that in most – but not all – cases, groups using EBS produce more ideas than groups using verbal brainstorming. In contrast, the results comparing groups using EBS to groups using nominal group brainstorming have been mixed: sometimes EBS groups produce more ideas, while in other cases, nominal groups produce more. This article examines the effects of group size on EBS, verbal brainstorming and nominal group brainstorming. We found that group size is a significant factor in predicting the performance of EBS relative to verbal brainstorming and nominal group brainstorming. As group size increases, the relative benefit of EBS increases. EBS groups outperform verbal groups when group size reaches four people. EBS groups outperform nominal groups when group size reaches 10 people.
A series of studies examined the role of social influence processes in group brainstorming. Two studies with pairs and 1 with groups of 4 revealed that the performance of participants in interactive groups is more similar than the performance of those in nominal groups. A 4th study demonstrated that performance levels in an initial group session predicted performance on a different problem 2 sessions later. In a 5th study it was found that the productivity gap between an interactive and nominal group could be eliminated by giving interactive group members a performance standard comparable with the typical performance of nominal groups. These studies indicate that performance levels in brainstorming groups are strongly affected by exposure to information about the performance of others. It is proposed that social matching of low performance levels by interactive group members may be an important factor in the productivity loss observed in group brain storming.
This study investigated the effect of brainstorming experience on the ability of groups to subsequently select the best ideas for implementation. Participants generated ideas either alone or in interactive groups and with either the regular brainstorming rules or with additional rules designed to further increase the number of ideas generated. All participants subsequently were asked to select their top five ideas in a group evaluation phase. Groups of individuals generating ideas in isolation (nominal groups) generated more ideas and more original ideas and were more likely to select original ideas during the group decision phase than interactive group brainstormers. Additional rules increased idea generation but not idea originality or idea selection.
Group creativity is the creation, development, evaluation, and promotion of novel ideas in groups. This can occur informally in interactions among friends or colleagues or in more structured groups such as scientific research laboratories and research and development teams. However, research has found a number of factors that limit the creative potential of groups such as a focus on being agreeable, concern about evaluation of ideas by others, and the difficulty of expressing one's ideas while others are expressing theirs. However, groups that vary in their expertise and perspectives should have great potential for creativity. Research studies and theoretical models have suggested a number of conditions that are critical for tapping this potential, such as effective use of dissent, appropriate leadership, and interaction modalities that reduce group interference.
This volume critically evaluates more than a century of empirical research on the effectiveness of small, task-performing groups, and offers a fresh look at the costs and benefits of collaborative work arrangements. The central question taken up by this book is whether -- and under what conditions -- interaction among group members leads to better performance than would otherwise be achieved simply by combining the separate efforts of an equal number of people who work independently. This question is considered with respect to a range of tasks (idea-generation, problem solving, judgment, and decision-making) and from several different process perspectives (learning and memory, motivation, and member diversity).