No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Effect of Peripheral Micro-tasks on Crowd Ideation
Abstract
Research has explored different ways of improving crowd ideation, such as presenting examples or employing facilitators. While such support is usually generated through peripheral tasks delegated to crowd workers who are not part of the ideation, it is possible that the ideators themselves could benefit from the extra thought involved in doing them. Therefore, we iterate over an ideation system in which ideators can perform one of three peripheral tasks (rating originality and usefulness, similarity, or idea combination) on demand. In controlled experiments with workers on Mechanical Turk, we compare the effects of these secondary tasks to simple idea exposure or no support at all, examining usage of the inspirations, fluency, breadth, and depth of ideas generated. We find tasks to be as good or better than exposure, although this depends on the period of ideation and the fluency level. We also discuss implications of inspiration size, homogeneity, and frequency.
... Finally, in our previous work we attempted to increase the effect of inspirations by adding a small task (e.g. rating the idea) to boost attention to the ideas [12]. ...
... The timing and delivery method of these inspirations can also affect their efficacy, showing benefits to giving users a choice of when to receive inspirations or in employing a smart strategy for choosing the right moment to do so [25]. Finally, increasing the attention to the inspirations ideas, such as by asking questions about the inspirations, can also improve performance under certain circumstances [12]. ...
... "rate the idea's originality and usefulness"). The microtasks are used to increase the attention to ideas and consequently their effect on ideators [12]. This mechanism is similar to other studies described in the literature, employing a pull model where inspirations are requested by ideators through the click of a button [4,12,25]. ...
Online crowds, with their large numbers and diversity, show great potential for creativity. Research has explored different ways of augmenting their creative performance, particularly during large-scale brainstorming sessions. Traditionally, this comes in the form of showing ideators some form of inspiration to get them to explore more categories or generate more and better ideas. The mechanisms used to select which inspirations are shown to ideators thus far have not taken into consideration ideators' individualities, which could hinder the effectiveness of support. In this paper, we introduce and evaluate CrowdMuse, a novel adaptive system for supporting large-scale brainstorming. The system models ideators based on their past ideas and adapts the system views and inspiration mechanism accordingly. We evaluate CrowdMuse over two iterative large online studies and discuss the implication of our findings for designing adaptive creativity support systems.
... However, it turned out that this approach requires a higher mental effort from the ideators. Girotto et al. propose similarity assessment by mixing human-based similarity evaluation with the ideation [4]. Each time users request an inspiration, they are asked to rate the similarity of one idea to three other ideas randomly obtained from all ideas. ...
... On the other hand, the impact of the validation task on the ideation outcome could be similar to a peripheral micro-task. According to Girotto et al. [4], microtasks encourage ideators to think about their ideas more deeply, mitigating the negative impact of the added complexity. We conducted a study to evaluate the impact of the validation task on the outcome of the ideation process. ...
... Following the methodology of related work [4], we compared the ideation outcome of our approach with two other conditions using a between subjects study design. We built on existing research in the baseline condition, by choosing an ideation system that provided inspiring examples to the ideators upon request. ...
A major goal of collaborative ideation is improving the creativity of the ideas generated. Recent approaches enhance creativity by showing users similar ideas during productive ideation and diverse ideas when they reach an impasse. However, related work either demands a higher mental effort from users to assess similarity or yields only a limited number of similarity values. Furthermore, idea relationship is only considered in one dimension (similarity). In our research in progress, we introduce a new approach called concept validation. It enables us to (1) capture the conceptualization of users' ideas and (2) assess multi-dimensional relationships between all ideas in near real-time. We conducted a study with 90 participants to validate the suitability of our approach. The results indicate that adding the extraction of semantic concepts to the ideation process has no negative impact on number and creativity of ideas generated. This signifies an important step towards our vision of an idea-based knowledge graph used by an interactive system to improve computer-supported human creativity.
... Dedicated online platforms have been developed for displaying the competitions, advertising, publishing, generating assigning and integrating the microtasks [15]- [17]. The microtasks are then published on the dedicated platforms to explore the best available crowd workers [18]. Moreover, the aims of dedicated online systems are to act as trustworthy intermediary platform and to resolve the disputes among clients and crowd workers [19], [20]. ...
... It has been noticed that published microtasks can be of technical (programming and development) as well as non-technical in nature [22]. Only a few noteworthy studies [18], [23], [24] have investigated different kinds and examples of microtasks which exist on web. However, their findings did not cover all the possible and existing microtasking areas. ...
With the utilization of crowdsourcing as a problem-solving approach, software industry has progressed tremendously in recent few years. Different models of crowdsourcing have been used depending on the nature of required outcome of the task, and have had varying levels of success to date. Microtasking is one of the lucrative models of crowdsourcing which penetrated the problem-solving strategy by facilitating the decomposition of complex tasks into short and self-contained microtasks which can be performed in few minutes. Regardless of considerable number of studies explored the kinds of microtasks, existing researches fall short when it comes to technical as well as non-technical tasks and the categorization of relevant microtasks. Thus, the aim of this research is to understand the context of microtasked related crowdsourcing and to explore the microtasks related to crowdsourced software development which exist in literature. Systematic literature review is conducted to identify the microtasking activities and expert review is conducted to validate the identified microtasking activities and their categories. The final publication sample to review the literature is composed of 42 research articles and the reviews of 4 experts are taken for validation. A total of 72 microtasking activities are found along with 11 categories. After validation applied, researchers came up with a list of 61 unique microtasking activities. This paper contributes to software industry by providing list of microtasks along with their categories which will be fruitful for researchers, microtasking platforms and their clients. It contributes to software industry by providing list of microtasks which will be fruitful for researchers and microtasking platforms.
... One of the directions taken in creativity research in HCI is to elicit design elements or requirements of creativity support and/or to develop creativity support tools (CST) using computer techniques to facilitate creative thinking [19,26,27,41,55,74]. Recently, a growing body of CST research has been adopting artifcial intelligence (AI) and focusing on AI-based interface development to model large-scale datasets and provide analytic insights to users in many design domains, such as app interfaces [19,71], graphics [47], and fashion [39,74]. ...
... In the following subsections, we will explain the algorithms used for StyleQ implementation. (14), garments parts (27), textile patterns (21), decorations (15), and textile fnishing (9). We collected a total of 25,470 fashion images from the Fashion14 dataset (12,190) [72] and by web crawling Google Images (13,280). ...
... However, the practice of such crowd-based ideation has revealed a significant limitation: Many ideas have a mundane, rudimentary and repetitive character [54,55]. Therefore, creativity-enhancing interventions were proposed as approaches to improve the quality of the ideas provided [6,24,[54][55][56]. A prerequisite for providing and evaluating creativity-enhancing interventions is an understanding of the meaning of ideas created by people. ...
... Furthermore, the influence of the ICV on the human ideation processes (their creativity) also needs to be captured. The operationalization of creativity in collaborative ideation is most often based on two metrics: Fluency [56] and dissimilarity of ideas [24]. Fluency describes the number of ideas produced by a participant. ...
Interactive intelligent systems, i.e., interactive systems that employ AI technologies, are currently present in many parts of our social, public and political life. An issue reoccurring often in the development of these systems is the question regarding the level of appropriate human and computer contributions. Engineers and designers lack a way of systematically defining and delimiting possible options for designing such systems in terms of levels of automation. In this paper, we propose, apply and reflect on a method for human-computer configuration design. It supports the systematic investigation of the design space for developing an interactive intelligent system. We illustrate our method with a use case in the context of collaborative ideation. Here, we developed a tool for information extraction from idea content. A challenge was to find the right level of algorithmic support, whereby the quality of the information extraction should be as high as possible, but, at the same time, the human effort should be low. Such contradicting goals are often an issue in system development; thus, our method proposed helped us to conceptualize and explore the design space. Based on a critical reflection on our method application, we want to offer a complementary perspective to the value-centered design of interactive intelligent systems. Our overarching goal is to contribute to the design of so-called hybrid systems where humans and computers are partners.
... However, the practice of such crowd-based ideation has revealed a significant limitation: Many ideas have a mundane, rudimentary and repetitive character [54,55]. Therefore, creativity-enhancing interventions were proposed as approaches to improve the quality of the ideas provided [6,24,[54][55][56]. A prerequisite for providing and evaluating creativity-enhancing interventions is an understanding of the meaning of ideas created by people. ...
... Furthermore, the influence of the ICV on the human ideation processes (their creativity) also needs to be captured. The operationalization of creativity in collaborative ideation is most often based on two metrics: Fluency [56] and dissimilarity of ideas [24]. Fluency describes the number of ideas produced by a participant. ...
Interactive intelligent systems, i.e., interactive systems that employ AI technologies, are currently present in many parts of our social, public and political life. An issue reoccurring often in the development of these systems is the question regarding the level of appropriate human and computer contributions. Engineers and designers lack a way of systematically defining and delimiting possible options for designing such systems in terms of levels of automation. In this paper, we propose, apply and reflect on a method for human-computer configuration design. It supports the systematic investigation of the design space for developing an interactive intelligent system. We illustrate our method with a use case in the context of collaborative ideation. Here, we developed a tool for information extraction from idea content. A challenge was to find the right level of algorithmic support, whereby the quality of the information extraction should be as high as possible, but, at the same time, the human effort should be low. Such contradicting goals are often an issue in system development; thus, our method proposed helped us to conceptualize and explore the design space. Based on a critical reflection on our method application, we want to offer a complementary perspective to the value-centered design of interactive intelligent systems. Our overarching goal is to contribute to the design of so-called hybrid systems where humans and computers are partners.
... The microtasks are used to increase the attention to ideas, following our previous work [6]. ...
... Explicit inspirations are shown when the user clicks the inspiration button, at which point the system selects and shows three ideas from other ideators (along with the microtask for each). This intervention is designed to strongly draw the ideator's attention to the ideas [6]. The selected ideas prioritize the categories that are likely to curb failures in generating new ideas within a category or in choosing a new category for idea generation [7]. ...
Online crowds, with their large numbers and diversity, show great potential for creativity, particularly during large-scale brainstorming sessions. Research has explored different ways of augmenting this creativity, such as showing ideators some form of inspiration to get them to explore more categories or generate more ideas. The mechanisms used to select which inspirations are shown to ideators thus far have been focused on characteristics of the inspirations rather than on ideators. This can hinder their effect, as creativity research has shown that ideators have unique cognitive structures and may therefore be better inspired by some ideas rather than others. We introduce CrowdMuse, an adaptive system for supporting large scale brainstorming. The system models ideators based on their past ideas and adapts the system views and inspiration mechanisms accordingly. An evaluation of this system could inform how to better individually support ideators.
... These systems can apply brainstorming techniques to reduce the problems such as peer pressure, social loafing, and production blocking. Besides brainstorming, some systems incorporate other creative techniques, such as peripheral micro tasks [24], to facilitate ideation. ...
Teams in tech companies collaboratively solve creative problems, and this team creative process increasingly occurs online. Despite many innovative tool designs to support collaborative creativity, many teams did not adopt them. This study aimed to identify the barriers to adoption and teams’ needs for collaborative creativity support. We clarified the team creative process in practice by individual in-depth interviews with 15 employees in 12 different high-tech companies in China. The results suggested that the teams frequently shared information via communication tools or face to face, and many of them acknowledged the benefits of collaboration in team creativity. However, most of them ideated and evaluated ideas or solutions individually. The reasons for the low collaboration level included the features of their tasks, a lack of technical and managerial support for collaboration in the ideation and evaluation phase, and a lack of motivation for team creativity. Based on these findings, we outlined implications for designing tools to support team creativity and demonstrated a prototype of a communication tool with creative support features.
... Crowd-sourced ideation has received much attention as it enables a larger number of participants to be gathered and a larger number of ideas to be generated [8][9][10]. Moreover crowd-sourced ideation enhances the diversity of the participants to include people related to the entire value chain of the products and services. ...
Brainstorming is an effective technique for offline ideation although the number of participants able to join an ideation session and suggest ideas is limited. To increase the diversity and quality of the ideas suggested, many participants with various backgrounds should be able to join the session. We have devised an evolutionary computation-assisted brainwriting method for large-scale online ideation. In this method, participants not only suggest ideas but also evaluate ideas previously suggested by other participants. The evaluation results are used in the evolutionary computation to identify good ideas to which the participants can be exposed via a brainwriting-like interface. We compared the performance of the proposed method with that of a simple online brainwriting method for large-scale online ideation with more than 30 participants. The proposed method enhanced robustness of idea quality improvement due to preferentially exposing the participants to good ideas.
... One of the directions taken in creativity research in HCI is to elicit design elements or requirements of creativity support and/or to develop creativity support tools (CST) using computer techniques to facilitate creative thinking [19,26,27,41,55,74]. Recently, a growing body of CST research has been adopting artificial intelligence (AI) and focusing on AI-based interface development to model large-scale datasets and provide analytic insights to users in many design domains, such as app interfaces [19,71], graphics [47], and fashion [39,74]. ...
Recent research on creativity support tools (CST) adopts artificial intelligence (AI) that leverages big data and computational capabilities to facilitate creative work. Our work aims to articulate the role of AI in supporting creativity with a case study of an AI-based CST tool in fashion design based on theoretical groundings. We developed AI models by externalizing three cognitive operations (extending, constraining, and blending) that are associated with divergent and convergent thinking. We present FashionQ, an AI-based CST that has three interactive visualization tools (StyleQ, TrendQ, and MergeQ). Through interviews and a user study with 20 fashion design professionals (10 participants for the interviews and 10 for the user study), we demonstrate the effectiveness of FashionQ on facilitating divergent and convergent thinking and identify opportunities and challenges of incorporating AI in the ideation process. Our findings highlight the role and use of AI in each cognitive operation based on professionals' expertise and suggest future implications of AI-based CST development.
... Year Publications 2018 [105,142,150,30,57,29,85,115,146,66] 2017 [90,5,2,152,45,164,58,79,123,120,136,99,109,36,95,139] 2016 [119,59,160,78,82,107,143,76,39,98,6,108,25,35,121] 2015 [51,38,159,126,140,144,97,84,112] 2014 [37,156,111,96,113,163,155,52,80,94,13,83] 2013 [158,81,104,14,157,162,9,114,16,62,40] 2012 [118,65,55,24,135] 2011 [161,103,149,141,41,89,69,56,28] 2010 [12,21,154,148,132,8,11,26,77,110,106] We hope that the insights from our survey and the recommendations based on our discussion will inspire the community to strengthen their efforts in addressing this challenge and thus identify new and established ways to evaluate CSTs. ...
... In a crowdsourced fashion, participants provide their ideas in individual time-bounded ideation sessions after reading the challenge. Related work has found that the effectiveness of inspirations depends on a person's cognitive state [22], the semantic similarity of the inspirations provided [3] and the level of attention a person devotes to an inspiration [7]. However, when summarizing the existing approaches of providing inspirations in large-scale ideation systems, we can surmise that these approaches disregard individual preferences people might have when using inspirations during ideation. ...
People react differently to inspirations shown to them during brainstorming. Existing research on large-scale ideation systems has investigated this phenomenon through aspects of timing, inspiration similarity and inspiration integration. However, these approaches do not address people's individual preferences. In the research presented, we aim to address this lack with regards to inspirations. In a first step, we conducted a co-located brainstorming study with 15 participants, which allowed us to differentiate two types of ideators: Inspiration seekers and inspiration avoiders. These insights informed the study design of the second step, where we propose a user model for classifying people depending on their ideator types, which was translated into a rule-based and a random forest-based classifier. We evaluated the validity of our user model by conducting an online experiment with 380 participants. The results confirmed our proposed ideator types, showing that, while seekers benefit from the availability of inspiration, avoiders were influenced negatively. The random forest classifier enabled us to differentiate people with a 73 \% accuracy after only three minutes of ideation. These insights show that the proposed ideator types are a promising user model for large-scale ideation. In future work, this distinction may help to design more personalized large-scale ideation systems that recommend inspirations adaptively.
... Juxtapoze [3] is a less constrained ap- proach applied to clipart composition, supporting serendipity and creative exploration with a shape-based search. Crowdbased approaches [2,19,20] incorporate ideation and feedback at large scale into specific parts or tasks in design problems. Recent machine learning and knowledge-based approaches include using topic modeling to cluster ideas based on diversity, quality, and representation, from repositories maintained by open innovation communities [1]. ...
Mind mapping is a popular way to explore a design space in creative thinking exercises, allowing users to form associations between concepts. Yet, most existing digital tools for mind mapping focus on authoring and organization, with little support for addressing the challenges of mind mapping such as stagnation and design fixation. We present Spinneret, a functional approach to aid mind mapping by providing suggestions based on a knowledge graph. Spinneret uses biased random walks to explore the knowledge graph in the neighborhood of an existing concept node in the mind map, and provides "suggestions" for the user to add to the mind map. A comparative study with a baseline mind-mapping tool reveals that participants created more diverse and distinct concepts with Spinneret, and reported that the suggestions inspired them to think of ideas they would otherwise not have explored.
... To improve creativity in this context, research in collaborative ideation has shown that computational support of individuals via the deployment of inspiring ideas of others shows great promise [2,3,6]. Although different modes of timing and content are analyzed in this area, there are two areas for improvement in this field: First, the existing approaches to creativity-enhancing interventions analyze ideas only on a statistical level. ...
To effectively support individual contributors in large scale ideation settings, we need a computational understanding of their cognitive processes. Although related work exists that references models from psychology, there are two shortcomings in existing approaches: First, they only analyse ideas on a statistical level. Second, they lack a notion of individual ideator differences. This work proposes a new user model for ideation, based on process-models from psychology and knowledge-graph based information extraction from submitted idea-texts. By building a computational model of ideation, this work aims to enable new graph-based analysis methods of cognitive ideation processes and new approaches to adaptive ideation support systems. These systems could be used to detect and counter fixation in individuals and to guide group efforts based on categories that should be explored.
... Moving brainstorming from a collocated setting to an open online platform yields several benefits: (1) a large number of the crowd allows the generation of a large number of ideas and (2) the heterogeneity of the crowd increases the potential for high quality ideas due to crowd's different background [11]. However, new challenges arise in distributed large-scale ideation platforms, for example, (a) many ideas are basic, mundane and repetitive [9] and (b) due to the number of ideas it's economically unfeasible to sift through all of them in order to filter-out low quality ideas [9]. ...
Crowd ideation platforms provide a promising approach for supporting the idea generation process. Research has shown that presenting a set of similar or diverse ideas to users during ideation leads them to come-up with more creative ideas. In this paper, we describe Innovonto, a crowd ideation platform that leverages semantic web technologies and human collaboration to identify similar and diverse ideas in order to enhance the creativity of generated ideas. Therefore, the approach implemented captures first the conceptualization of users' ideas. Then, a matching system is employed to compute similarities between all ideas in near real time. Furthermore, this technical report outlines the results obtained from the evaluation of Innovonto platform. The hallway study, conducted at our research group, allowed us to test each step of Innovonto platform as well as the proposed approach in assessing similarities between ideas. As results, we received 20 ideas and 23 feedback from 9 users. The analysis of the results shows good performance of Innovonto steps and confirms the findings of existing research.
The COVID-19 pandemic led to dire consequences globally, and it has been particularly challenging for older adults. They are at a higher risk of adverse outcomes of the disease \citecdc_covid. Older adults also use less technology than other age groups \citetech_old, so they mostly rely on in-person interactions and services for social support. However, disease mitigation efforts such as social distancing and self-quarantining severely limited in-person interactions, hindering older adults' social support during the COVID-19 crisis. In this paper, we present findings on social support realities from semi-structured interviews with older adults (N=15) living alone in community dwellings. We found that older adults' support roles, support sources, and support concerns evolve as they passed through this time of sweeping change. They are enthusiastic about providing support to people who are older and more vulnerable than themselves. At the same time, their needs for safety, autonomy, and independence create tensions around social support. We propose a framework to illustrate the evolving ecology of social support that can facilitate the holistic design of socio-technical support systems for older adults. We argue against the societal portrayal of older adults as vulnerable individuals. Rather, there is an opportunity to design support systems considering them as anchors in society. Towards that goal, we present design implications for future socio-technical support systems to empower older adults to age in place during a crisis.
Creating truly original ideas requires extensive knowledge of existing ideas. Navigating prior examples can help people to understand what has already been done and to assess the quality of their own ideas through comparison. The creativity literature has suggested that the conceptual distance between a proposed solution and a potential inspiration can influence one's thinking. However, less is known about how creators might use data about conceptual distance when exploring a large repository of ideas. To investigate this, we created a novel tool for exploring examples called IdeateRelate that visualizes 600+ COVID-related ideas, organized by their similarity to a new idea. In an experiment that compared the IdeateRelate visualization to a simple list of examples, we found that users in the Viz condition leveraged both semantic and categorical similarity, curated a more similar set of examples, and adopted more language from examples into their iterated ideas (without negatively affecting the overall novelty). We discuss implications for creating adaptive interfaces that provide creative inspiration in response to designers' ideas throughout an iterative design process.
Creativity Support Tools (CSTs) play a fundamental role in the study of creativity in Human-Computer Interaction (HCI). Even so, there is no consensus definition of the term 'CST' in HCI, and in most studies, CSTs have been construed as one-off exploratory prototypes, typically built by the researchers themselves. This makes it difficult to clearly demarcate CST research, but also to compare findings across studies, which impedes advancement in digital creativity as a growing field of research. Based on a literature review of 143 papers from the ACM Digital Library (1999-2018), we contribute a first overview of the key characteristics of CSTs developed by the HCI community. Moreover, we propose a tentative definition of a CST to help strengthen knowledge sharing across CST studies. We end by discussing our study's implications for future HCI research on CSTs and digital creativity.
Enhancing creativity has been paid a lot of attention recently, especially with the emergence of online collaborative ideation. Prior work has shown that, in addition to the exposure of diverse and creative examples, visualising the solution space enables ideators to be inspired and, thus, arrive at more creative ideas. However, existing automated approaches which assess the diversity of a set of examples fail on unstructured short text due to their reliance on similarity computation. Furthermore, the conceptual divergence cannot be easily captured for such representations. This research in progress introduces an approach based on semantic annotation to overcome these issues. The solution proposed formalizes user ideas into a set of annotated concepts and a matching mechanism is then used to compute the similarity between users' ideas. We aim also to create a visualisation of the solution space based on the similarity matrix obtained by a matching process between all ideas.
Research on group brainstorming has demonstrated that it is less effective for generating large numbers of ideas than individual brainstorming, yet various scholars have presumed that group idea sharing should enhance cognitive stimulation and idea production. Three experiments examined the potential of cognitive stimulation in brainstorming. Experiments 1 and 2 used a paradigm in which individuals were exposed to ideas on audiotape as they were brainstorming, and Experiment 3 used the electronic brainstorming paradigm. Evidence was obtained for enhanced idea generation both during and after idea exposure. The attentional set of the participant and the content of the exposure manipulation (number of ideas, presence of irrelevant information) influenced this effect. These results are consistent with a cognitive perspective on group brainstorming.
This paper reports on a series of experiments which were conducted to test the hypothesis that design fixation, defined as a blind adherence to a set of ideas or concepts limiting the output of conceptual design, is a measurable barrier in the conceptual design process. The results of the experiments clearly demonstrate the existence of design fixation. The paper related issues such as the nature of the phenomenon, some experimental issues which arise in such investigations, and directions for future research.
The Cambridge Handbook of Creativity is a comprehensive scholarly handbook on creativity from the most respected psychologists, researchers and educators. This handbook serves both as a thorough introduction to the field of creativity and as an invaluable reference and current source of important information. It covers such diverse topics as the brain, education, business, and world cultures. The first section, 'Basic Concepts', is designed to introduce readers to both the history of and key concepts in the field of creativity. The next section, 'Diverse Perspectives of Creativity', contains chapters on the many ways of approaching creativity. Several of these approaches, such as the functional, evolutionary, and neuroscientific approaches, have been invented or greatly reconceptualized in the last decade. The third section, 'Contemporary Debates', highlights ongoing topics that still inspire discussion. Finally, the editors summarize and discuss important concepts from the book and look to what lies ahead.
Researchers of group creativity have noted problems such as social loafing, production blocking, and especially, evaluation apprehension. Thus, brainstorming techniques have specifically admonished people 'not to criticize' their own and others' ideas, a tenet that has gone unexamined. In contrast, there is research showing that dissent, debate and competing views have positive value, stimulating divergent and creative thought. Perhaps more importantly, we suggest that the permission to criticize and debate may encourage an atmosphere conducive to idea generation. In this experimental study, traditional brainstorming instructions, including the advice of not criticizing, were compared with instructions encouraging people to debate—even criticize. A third condition served as a control. This study was conducted both in the United States and in France. Results show the value of both types of instruction, but, in general, debate instructions were superior to traditional brainstorming instruc- tions. Further, these findings hold across both cultures. Results are discussed in terms of the potential positive value of encouraging debate and controversy for idea generation. Copyright # 2004 John Wiley & Sons, Ltd.
Two experiments were conducted to explore the process of building on ideas in brainstorming. Although this is presumed to be an important role of brainstorming, this has never been explored experimentally. In one experiment individual and group brainstormers generated ideas which were subsequently presented to these same individuals and groups to combine and build on for additional ideas, either as groups or individuals. The combination process was influenced by whether the participants had previously brainstormed alone or in groups and the phase of the combination period (early vs. late). In a second study participants were presented lists of rare or common ideas to combine and build on either as individuals or groups. Although groups generated fewer combinations than nominal groups, they generated more novel and feasible combinations when combining rare ideas. These findings indicate that groups are able to benefit from the exchange process in building on each other's ideas and are interpreted in the context of past research on idea generation and evaluation in groups.
Abstract ,The first ideas to be considered during creative idea generation can have profoundly constraining effects on the scope of the ideas that are subsequently generated. Even if initial ideasare intended to serve as helpful examples, or they are given simply to get the creative process going, the constraints of initial ideas may,be inescapable. Such constraints can impede,successful problem,solving and inhibit creative invention. Overcoming these constraints can be enhanced by reconsidering initially failed problems in new contexts. Empirical research examining,cognitive mechanisms,for these constraints is discussed. ,3 The Constraining Effects of Initial Ideas ,What is the best source of creative ideas? The wisdom of proverbs advises us that if we are to see farther than others, wemust "stand on the shoulders of giants." This means,thatwe,should use the prior knowledge,that has been provided by our predecessors, because in solving problems there is no need to "re-invent the wheel." On the other hand, a different proverb advises us not to get "stuck in a rut," meaning that using prior knowledge to solve problems can lead us to the same,old tired ideas and blocked thinking that stymied progress on those problems,in the past. How are we to choose,between,these apparently contradictory sources of wisdom? One way to address this question is through empirical studies of creative thinking and problem solving, a method
Free production of variability through unfettered divergent thinking holds out the seductive promise of effortless creativity, but runs the risk of generating only quasi-creativity or pseudo-creativity if it is not adapted to reality. Thus, creative thinking seems to involve two components: generation of novelty (via divergent thinking) and evaluation of the novelty (via convergent thinking). In the area of convergent thinking, knowledge is of particular importance: It is a source of ideas, suggests pathways to solutions, and provides criteria of effectiveness and novelty. The way in which the two kinds of thinking work together can be understood in terms of thinking styles or of phases in the generation of creative products. In practical situations, divergent thinking without convergent thinking can cause a variety of problems including reckless change. None the less, care must be exercised by those who sing the praises of convergent thinking: Both too little and too much is bad for creativity.
We conducted four experiments to investigate free riding, evaluation apprehension, and production blocking as explanations of the difference in brainstorming productivity typically observed between real and nominal groups. In Experiment 1, we manipulated assessment expectations in group and individual brainstorming. Although productivity was higher when subjects worked under personal rather than collective assessment instructions, type of session still had a major impact on brainstorming productivity under conditions that eliminated the temptation to free ride. Experiment 2 demonstrated that inducing evaluation apprehension reduced productivity in individual brainstorming. However, the failure to find an interaction between evaluation apprehension and type of session in Experiment 3 raises doubts about evaluation apprehension as a major explanation of the productivity loss in brainstorming groups. Finally, by manipulating blocking directly, we determined in Experiment 4 that production blocking accounted for most of the productivity loss of real brainstorming groups. The processes underlying production blocking are discussed, and a motivational interpretation of blocking is offered. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Research has consistently found nominal group brainstorming (in which members work separately without communicating) to be superior to brainstorming in which group members interact verbally. This article presents the results of an experiment that found the reverse to be true for computer-mediated electronic brainstorming. In this experiment, 12-member electronically interacting groups generated more ideas than did 12-member nominal groups, and there were no differences between 6-member electronic and 6-member nominal groups. The authors attribute these results to the ability of electronic brainstorming to introduce few process losses (production blocking, evaluation apprehension, and free riding) while enabling process gains (synergy and the avoidance of redundant ideas). (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Considers the definition and assessment of creativity and presents a componential framework for conceptualizing this faculty. Including domain-relevant skills, creativity-relevant skills, and task motivation as a set of necessary and sufficient components of creativity, the framework describes the way in which cognitive abilities, personality characteristics, and social factors might contribute to stages of the creative process. The discussion emphasizes the previously neglected social factors and highlights the contributions that a social psychology of creativity can make to a comprehensive view of creative performance. (99 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Three experiments were performed to test Smith, Ward, and Schumacher’s (1993) conformity hypothesis— that people’s ideas will
conform to examples they are shown in a creative generation task. Conformity was observed in all three experiments; participants
tended to incorporate critical features of experimenter-provided examples. However, examination of total output, elaborateness
of design, and the noncritical features did not confirm that the conformity effect constrained creative output in any of the
three experiments. Increasing the number of examples increased the conformity effect (Experiment 1). Examples that covaried
features that are naturally uncorrelated in the real world led to a greater subjective rating of creativity (Experiment 2).
A delay between presentation and test increased conformity (Experiment 3), just as models of inadvertent plagiarism would
predict. The explanatory power of theoretical accounts such as activation, retrieval blocking, structured imagination, and
category abstraction are evaluated.
The effects of idea sharing on cognitive processes and performance were assessed in an idea exposure paradigm. Participants generated ideas while being exposed to stimulus ideas that were semantically homogeneous or diverse, and were offered in an organized or a random sequence. As compared to a control condition, participants generated more diverse ideas when exposed to ideas from a wide range of semantic categories, and they generated more ideas per category when exposed to many ideas from only a few categories. The semantic organization of ideas was higher when participants were exposed to ideas that were organized in semantic clusters than when participants were exposed to unorganized ideas. Idea exposure had positive effects in general, because it reduced response latencies for category changes. Implications for information processing in groups are discussed.
A sketch combination system is introduced and tested: a crowd of 1047 participated in an iterative process of design, evaluation and combination. Specifically, participants in a crowdsourcing marketplace sketched chairs for children. One crowd created a first generation of chairs, and then successive crowds created new generations by combining the chairs made by previous crowds. Other participants evaluated the chairs. The crowd judged the chairs from the third generation more creative than those from the first generation. An analysis of the design evolution shows that participants inherited and modified presented features, and also added new features. These findings suggest that crowd based design processes may be effective, and point the way toward computer-human interactions that might further encourage crowd creativity.
User studies are important for many aspects of the design process and involve techniques ranging from informal surveys to rigorous laboratory studies. However, the costs involved in engaging users often requires practitioners to trade off between sample size, time requirements, and monetary costs. Micro-task markets, such as Amazon's Mechanical Turk, offer a potential paradigm for engaging a large number of users for low time and monetary costs. Here we investigate the utility of a micro-task market for collecting user measurements, and discuss design considerations for developing remote micro user evaluation tasks. Although micro-task markets have great potential for rapidly collecting user measurements at low costs, we found that special care is needed in formulating tasks in order to harness the capabilities of the approach. Author Keywords Remote user study, Mechanical Turk, micro task, Wikipedia.
Micro-task markets such as Amazon's Mechanical Turk represent a new paradigm for accomplishing work, in which employers can tap into a large population of workers around the globe to accomplish tasks in a fraction of the time and money of more traditional methods. However, such markets have been primarily used for simple, independent tasks, such as labeling an image or judging the relevance of a search result. Here we present a general purpose framework for accomplishing complex and interdependent tasks using micro-task markets. We describe our framework, a web-based prototype, and case studies on article writing, decision making, and science journalism that demonstrate the benefits and limitations of the approach.
Electronic brainstorming systems have been shown to lead to more ideas, yet unsupported face-to-face brainstorming is still widely preferred. This pa- per proposes a graphical user interface for a web-based system for design problem-solving or other intellective tasks involving convergent and divergent thinking. Referring to the literature on group support systems and information and knowledge visualization, the study extends features of concept mapping and synthesizes these into a prototype called the Concept Plot (CP). Based on an advertising design task, the paper shows how the CP can be collabo- ratively constructed in two directions, as text and pictures are uploaded onto nodes, and these nodes scaled up or down as users click to evaluate ideas. The expectation is that this integrated visualization would diminish information overload, while enhancing the social dynamics of the process. Also presented is the pilot deployment of a Flash prototype. The results were inconclusive, yet promising that a study with more participants might demonstrate the func- tional and affective benefits of the CP. Information Visualization (2006) 5, 185--191. doi:10.1057/palgrave.ivs.9500130
Research on group brainstorming has demonstrated that it is less effective for generating large numbers of ideas than individual brainstorming, yet various scholars have presumed that group idea sharing should enhance cognitive stimulation and idea production. Three experiments examined the potential of cognitive stimulation in brainstorming. Experiments 1 and 2 used a paradigm in which individuals were exposed to ideas on audiotape as they were brainstorming, and Experiment 3 used the electronic brainstorming paradigm. Evidence was obtained for enhanced idea generation both during and after idea exposure. The attentional set of the participant and the content of the exposure manipulation (number of ideas, presence of irrelevant information) influenced this effect. These results are consistent with a cognitive perspective on group brainstorming.
A model called search for ideas in associative memory (SIAM) is proposed to account for various research findings in the area of group idea generation. The model assumes that idea generation is a repeated search for ideas in associative memory, which proceeds in 2 stages (knowledge activation and idea production), and is controlled through negative feedback loops and cognitive failures (trials in which no idea is generated). We show that (a) turn taking (production blocking) interferes with both stages of the process; (b) ideas suggested by others aid the activation of problem-relevant knowledge; and (c) cognitive failures are important determinants of brainstorming persistence, satisfaction, and enjoyment. Implications for group decision making and group recall are discussed.
Online crowds are a promising source of new innovations. However, crowd innovation quality does not always match its quantity. In this paper, we explore how to improve crowd innovation with real-time expert guidance. One approach would for experts to provide personalized feed-back, but this scales poorly, and may lead to premature convergence during creative work. Drawing on strategies for facilitating face-to-face brainstorms, we introduce a crowd ideation system where experts monitor incoming ideas through a dashboard and offer high-level “inspirations” to guide ideation. A series of controlled experiments show that experienced facilitators increased the quantity and creativity of workersâ ideas compared to unfacilitated workers, while Novice facilitators reduced workersâ creativity. Analyses of inspiration strategies suggest these opposing results stem from differential use of successful inspiration strategies (e.g., provoking mental simulations). The results show that expert facilitation can significantly improve crowd innovation, but inexperienced facilitators may need scaffolding to be successful.
Prior work on creativity support tools demonstrates how a computational semantic model of a solution space can enable interventions that substantially improve the number, quality and diversity of ideas. However, automated semantic modeling often falls short when people contribute short text snippets or sketches. Innovation platforms can employ humans to provide semantic judgments to construct a semantic model, but this relies on external workers completing a large number of tedious micro tasks. This requirement threatens both accuracy (external workers may lack expertise and context to make accurate semantic judgments) and scalability (external workers are costly). In this paper, we introduce IdeaHound, an ideation system that seamlessly integrates the task of defining semantic relationships among ideas into the primary task of idea generation. The system combines implicit human actions with machine learning to create a computational semantic model of the emerging solution space. The integrated nature of these judgments allows IDEAHOUND to leverage the expertise and efforts of participants who are already motivated to contribute to idea generation, overcoming the issues of scalability inherent to existing approaches. Our results show that participants were equally willing to use (and just as productive using) IDEAHOUND compared to a conventional platform that did not require organizing ideas. Our integrated crowdsourcing approach also creates a more accurate semantic model than an existing crowdsourced approach (performed by external crowds). We demonstrate how this model enables helpful creative interventions: providing diverse inspirational examples, providing similar ideas for a given idea and providing a visual overview of the solution space.
Large-scale idea generation platforms often expose ideators to previous ideas. However, research suggests people generate better ideas if they see abstracted solution paths (e.g., descriptions of solution approaches generated through human sensemaking) rather than being inundated with all prior ideas. Automated and semi-automated methods can also offer interpretations of earlier ideas. To benefit from sensemaking in practice with limited resources, ideation platform developers need to weigh the cost-quality tradeoffs of different methods for surfacing solution paths. To explore this, we conducted an online study where 245 participants generated ideas for two problems in one of five conditions: 1) no stimuli, 2) exposure to all prior ideas, or solution paths extracted from prior ideas using 3) a fully automated workflow, 4) a hybrid human-machine approach, and 5) a fully manual approach. Contrary to expectations, human-generated paths did not improve ideation (as meas-ured by fluency and breadth of ideation) over simply showing all ideas. Machine-generated paths sometimes significantly improved fluency and breadth of ideation over no ideas (although at some cost to idea quality). These findings suggest that automated sensemaking can improve idea generation, but we need more research to understand the value of human sensemaking for crowd ideation.
A growing number of large collaborative idea generation platforms promise that by generating ideas together, people can create better ideas than any would have alone. But how might these platforms best leverage the number and diversity of contributors to help each contributor generate even better ideas? Prior research suggests that seeing particularly creative or diverse ideas from others can inspire you, but few scalable mechanisms exist to assess diversity. We contribute a new scalable crowd-powered method for evaluating the diversity of sets of ideas. The method relies on similarity comparisons (is idea A more similar to B or C) generated by non-experts to create an abstract spatial idea map. Our validation study reveals that human raters agree with the estimates of dissimilarity derived from our idea map as much or more than they agree with each other. People seeing the diverse sets of examples from our idea map generate more diverse ideas than those seeing randomly selected examples. Our results also corroborate findings from prior research showing that people presented with creative examples generated more creative ideas than those who saw a set of random examples. We see this work as a step toward building more effective online systems for supporting large scale collective ideation.
Emerging online ideation platforms with thousands of example ideas provide an important resource for creative production. But how can ideators best use these examples to create new innovations? Recent work has suggested that not just the choice of examples, but also the timing of their delivery can impact creative outcomes. Building on existing cognitive theories of creative insight, we hypothesize that people are likely to benefit from examples when they run out of ideas. We explore two example delivery mechanisms that test this hypothesis: 1) a system that proactively provides examples when a user appears to have run out of ideas, and 2) a system that provides examples when a user explicitly requests them. Our online experiment (N=97) compared these two mechanisms against two baselines: providing no examples and automatically showing examples at a regular interval. Participants who requested examples themselves generated ideas that were rated the most novel by external evaluators. Participants who received ideas automatically when they appeared to be stuck produced the most ideas. Importantly, participants who received examples at a regular interval generated fewer ideas than participants who received no examples, suggesting that mere access to examples is not sufficient for creative inspiration. These results emphasize the importance of the timing of example delivery. Insights from this study can inform the design of collective ideation support systems that help people generate many high quality ideas.
The Cambridge Handbook of Creativity is a comprehensive scholarly handbook on creativity from the most respected psychologists, researchers and educators. This handbook serves both as a thorough introduction to the field of creativity and as an invaluable reference and current source of important information. It covers such diverse topics as the brain, education, business, and world cultures. The first section, 'Basic Concepts', is designed to introduce readers to both the history of and key concepts in the field of creativity. The next section, 'Diverse Perspectives of Creativity', contains chapters on the many ways of approaching creativity. Several of these approaches, such as the functional, evolutionary, and neuroscientific approaches, have been invented or greatly reconceptualized in the last decade. The third section, 'Contemporary Debates', highlights ongoing topics that still inspire discussion. Finally, the editors summarize and discuss important concepts from the book and look to what lies ahead.
Seeking solutions from one domain to solve problems in another is an effective process of innovation. This process of analogy searching is difficult for both humans and machines. In this paper, we present a novel approach for re-presenting a problem in terms of its abstract structure, and then allowing people to use this structural representation to find analogies. We propose a crowdsourcing process that helps people navigate a large dataset to find analogies. Through two experiments, we show the benefits of using abstract structural representations to search for ideas that are analogous to a source problem, and that these analogies result in better solutions than alternative approaches. This work provides a useful method for finding analogies, and can streamline innovation for both novices and professional designers.
Harnessing crowds can be a powerful mechanism for increasing innovation. However, current approaches to crowd innovation rely on large numbers of contributors generating ideas independently in an unstructured way. We introduce a new approach called distributed analogical idea generation, which aims to make idea generation more effective and less reliant on chance. Drawing from the literature in cognitive science on analogy and schema induction, our approach decomposes the creative process in a structured way amenable to using crowds. In three experiments we show that distributed analogical idea generation leads to better ideas than example-based approaches, and investigate the conditions under which crowds generate good schemas and ideas. Our results have implications for improving creativity and building systems for distributed crowd innovation.
Taxonomies are a useful and ubiquitous way of organizing information. However, creating organizational hierarchies is difficult because the process requires a global understanding of the objects to be categorized. Usually one is created by an individual or a small group of people working together for hours or even days. Unfortunately, this centralized approach does not work well for the large, quickly changing datasets found on the web. Cascade is an automated workflow that allows crowd workers to spend as little at 20 seconds each while collectively making a taxonomy. We evaluate Cascade and show that on three datasets its quality is 80-90% of that of experts. Cascade has a competitive cost to expert information architects, despite taking six times more human labor. Fortunately, this labor can be parallelized such that Cascade will run in as fast as four minutes instead of hours or days.
Idea generation is an important step in the engineering design process, and as a result significant research efforts have focused on developing methods to aid designers in exploring design possibilities. Metrics to evaluate design exploration are thus necessary to make conclusions and comparisons among idea generation methods. Metrics have previously been proposed, identifying novelty, variety, quantity, and quality to characterize sets of designs and the degree to which they describe design space exploration. This article describes flaws in the variety metric and proposes a new metric to eliminate the flaws. Additionally, a single metric is proposed to evaluate the quality of design space exploration during concept generation, enabling application of a single metric to compare idea generation processes and methodologies.
This paper introduces architectural and interaction patterns for integrating crowdsourced human contributions directly into user interfaces. We focus on writing and editing, complex endeavors that span many levels of conceptual and pragmatic activity. Authoring tools offer help with pragmatics, but for higher-level help, writers commonly turn to other people. We thus present Soylent, a word processing interface that enables writers to call on Mechanical Turk workers to shorten, proofread, and otherwise edit parts of their documents on demand. To improve worker quality, we introduce the Find-Fix-Verify crowd programming pattern, which splits tasks into a series of generation and review stages. Evaluation studies demonstrate the feasibility of crowdsourced editing and investigate questions of reliability, cost, wait time, and work time for edits.
This chapter builds on some 15 years of research on group idea generation carried out at the University of Tübingen, Germany, and Utrecht University, The Netherlands. During these years, three questions have been the primary focus of attention: How can the productivity loss of brainstorming groups be explained? How can productivity of these groups be improved? Why do people think that group brainstorming is more effective than individual brainstorming, whereas the opposite is true? However, the research program has increasingly shifted toward a cognitive approach. The chapter is organized as follows. The first section describes some early research pertaining to the productivity loss of groups, possibilities for improving group performance, and the illusory belief that group brainstorming is more effective than individual brainstorming. It then outlines a cognitive approach to these issues and argues that earlier findings can be interpreted as either cognitive interference or cognitive stimulation effects. The third section discusses recent studies aimed at testing these ideas. The final section draws some conclusions and suggests practical implications and future directions for research.
The psychological study of creativity is essential to human progress. If strides are to be made in the sciences, humanities, and arts, we must arrive at a far more detailed understanding of the creative process, its antecedents, and its inhibitors. This review, encompassing most subspecialties in the study of creativity and focusing on twenty-first-century literature, reveals both a growing interest in creativity among psychologists and a growing fragmentation in the field. To be sure, research into the psychology of creativity has grown theoretically and methodologically sophisticated, and researchers have made important contributions from an ever-expanding variety of disciplines. But this expansion has not come without a price. Investigators in one subfield often seem unaware of advances in another. Deeper understanding requires more interdisciplinary research, based on a systems view of creativity that recognizes a variety of interrelated forces operating at multiple levels.
Methods and models for the quantitative analysis of crowd brainstorming
- Filip Krynicki
- Krynicki Filip
Filip Krynicki. 2014. Methods and models for the
quantitative analysis of crowd brainstorming. Retrieved
April
5,
2016
from
https://uwspace.uwaterloo.ca/handle/10012/8347
Theories of creativity. The Cambridge handbook of creativity
- Ronald A Aaron Kozbelt
- Beghetto
- A Mark
- Runco
The constraining effects of initial ideas In Group creativity: Innovation through collaboration
- M Steven
- Smith
Cascade: Crowdsourcing taxonomy creation
- James A Weld
- Landay
Weld, and James A. Landay. 2013. Cascade: Crowdsourcing
taxonomy creation. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, 19992008.
Retrieved
January
11,
2016
from
http://dl.acm.org/citation.cfm?id=2466265
Theories of creativity. The Cambridge handbook of creativity
- Runco
Runco. 2010. Theories of creativity. The Cambridge
handbook of creativity: 20-47.
Toward Collaborative Ideation at Scale: Leveraging Ideas from Others to Generate More Creative and Diverse Ideas
- Kenneth C Pao Siangliulue
- Z Arnold Krzysztof
- Steven P Gajos
- Dow
Gajos, and Steven P. Dow. 2015. Toward Collaborative
Ideation at Scale: Leveraging Ideas from Others to Generate
More
Creative
and
Diverse
Ideas.
937-945.