Crowd Science

Information technology is being increasingly employed to harness under-utilized resources via more effective coordination. This progress has manifested in different developments, for instance, crowdsourcing (e.g. Wikipedia, Amazon Mechanical Turk, and Waze), crowdfunding (e.g. Kickstarter, Indiegogo, and RocketHub) or the sharing economy (e.g. Uber, Airbnb, and Didi Chuxing). Since the sustainability of these IT-enabled forms of resource coordination do not commonly rely merely on direct economic benefits of the participants, but also on other non-monetary, intrinsic gratifications, such systems are increasingly gamified that is, designers use features of games to induce enjoyment and general autotelicy of the activity. However, a key problem in gamification design has been whether it is better to use competition-based or cooperation-based designs. We examine this question through a field experiment in a gamified crowdsourcing system, employing three versions of gamification: competitive, cooperative, and inter-team competitive gamification. We study these gamified conditions’ effects on users’ perceived enjoyment and usefulness of the system as well as on their behaviors (system usage, crowdsourcing participation, engagement with the gamification feature, and willingness to recommend the crowdsourcing application). The results reveal that inter-team competitions are most likely to lead to higher enjoyment and crowdsourcing participation, as well as to a higher willingness to recommending a system. Further, the findings indicate that designers should consider cooperative instead of competitive approaches to increase users’ willingness to recommend crowdsourcing systems. These insights add relevant findings to the ongoing discourse on the roles of different types of competitions in gamification designs and suggest that crowdsourcing system designers and operators should implement gamification with competing teams instead of typically used competitions between individuals.

In this work we seek to understand how differences in location affect participation outcomes in IT-mediated crowds. To do so, we operationalize Crowd Capital Theory with data from a popular international creative crowdsourcing site, to determine whether regional differences exist in crowdsourcing participation outcomes. We present the early results of our investigation from data encompassing 1,858,202 observations from 28,214 crowd members on 94 different projects in 2012. Using probit regressions to isolate geographic effects by continental region, we find significant variation across regions in crowdsourcing participation. In doing so, we contribute to the literature by illustrating that geography matters in respect to crowd participation. Further, our work illustrates an initial validation of Crowd Capital Theory as a useful theoretical model to guide empirical inquiry in the fast-growing domain of IT-mediated crowds.

Continued technological progress has placed IT at the epicenter of new markets (Archak & Sundararajan 2009), spawning a significant reduction in traditional market search and market coordination costs (Malone, Yates & Benjamin 1987, Bakos 1997). In this work, premised on the observation that virtual labor markets (VLMs) can generally be engaged through two different modes of IT (i.e. Dashboard vs. API), this exploratory work begins to unpack the ramifications of these material differences for Crowdsourcing market function.

Education is one of the eight Millennium Development Goals (MDG) of the United Nations. Considerable interest has been displayed in online education at scale, a new arising concept to realize the MDG. Yet connecting online education to real jobs is still a challenge. This CHI workshop bridges this gap by bringing together groups and insights from related work at HCOMP, CSCW, and Learning at Scale. The workshop aims at providing opportunities for groups not yet in the focus of online education, exemplified by low SES and less educated students who have not have equal access to higher education, compared to typical students in MOOCs. The focus is on theoretical and empirical connections between online education and job opportunities which can reduce the financial gap, by providing students with an income during their studies. The workshop explores the technological analogue of the concept of "apprenticeship", long established in the European Union, and education research (Collins, Seely Brown, Newman, 1989). This allows students to do useful work as an apprentice during their studies. This workshop tackles such questions by bringing together participants from industry (e.g., platforms similar to Upwork, Amazon Mechanical Turk); education, psychology, and MOOCs (e.g., attendees of AERA, EDM, AIED, Learning at Scale); crowdsourcing and collaborative work (e.g., attendees of CHI, CSCW, NIPS, AAAI's HCOMP).

We are seeing more and more organizations undertaking activities to engage dispersed populations through IS. Using the knowledge-based view of the organization, this work conceptualizes a theory of Crowd Capital to explain this phenomenon. Crowd Capital is a heterogeneous knowledge resource generated by an organization, through its use of Crowd Capability, which is defined by the structure, content, and process by which an organization engages with the dispersed knowledge of individuals (the Crowd). Our work draws upon a diverse literature and builds upon numerous examples of practitioner implementations to support our theorizing. We present a model of Crowd Capital generation in organizations and discuss the implications of Crowd Capital on organizational boundary and on IS research.

In this work we use the theory of Crowd Capital as a lens to compare and contrast a number of IS tools currently in use by organizations for crowd-engagement purposes. In doing so, we contribute to both the practitioner and research domains. For the practitioner community we provide decision-makers with a convenient and useful resource, in table-form, outlining in detail some of the differing potentialities of crowd-engaging IS. For the research community we begin to unpack some of the key properties of crowd-engaging IS, including some of the differing qualities of the crowds that these IS application engage.

Crowdsourcing is beginning to be used for policymaking. The “wisdom of crowds” [Surowiecki 2005], and crowdsourcing [Brabham 2008], are seen as new avenues that can shape all kinds of policy, from transportation policy [Nash 2009] to urban planning [Seltzer and Mahmoudi 2013], to climate policy. In general, many have high expectations for positive outcomes with crowdsourcing, and based on both anecdotal and empirical evidence, some of these expectations seem justified [Majchrzak and Malhotra 2013]. Yet, to our knowledge, research has yet to emerge that unpacks the different forms of crowdsourcing in light of each stage of the well-established policy cycle. This work addresses this research gap, and in doing so brings increased nuance to the application of crowdsourcing techniques for policymaking.

How does the geographic location of individual crowd members effect crowdsourcing participation and outcomes? How do variations in contest design effect crowdsourcing participation and outcomes by country and region? In this work we begin to answer these important questions by empirically testing the effects of geography and contest design on crowdsourcing participation and outcomes. Using data from a global creative crowdsourcing site, we utilize Crowd Capital Theory with data encompassing 1,858,202 observations from 28,214 crowd members on 94 different projects, to test our hypotheses based on the premise that the crowdsourcing is – much like the real world – not a flat one. Using multiple probit regressions to isolate the geographic effects, we find significant variation across countries and regions on crowdsourcing participation and outcomes, and further, significant effects of contest design on participation and outcomes by country and region. Our work makes new and useful contributions to the literature on crowdsourcing and creative competitions used for open innovation, ascertaining that the world is still not flat — even in case of tournaments on IT platforms with global reach. This is also the first work to empirically measure the distinct stages of resource creation from IT-mediated crowds; in doing so we validate Crowd Capital Theory as a model for resource creation from IT-mediated crowds.

In this work, we present a high-level computational model of IT-mediated crowds for collective intelligence. We introduce the Crowd Capital perspective as an organizational-level model of collective intelligence generation from IT-mediated crowds, and specify a computational system including agents, forms of IT, and organizational knowledge. As far as we are aware, this work, and our simulation software, represents the first attempt to computationally model the organizational knowledge generating capacity of IT-mediated crowds. Further, as far as we know, the work is also the first of its kind to computationally model episodic and collaborative IT structures, distinguishing those forms of IT that require crowd collaboration for knowledge generation, from those that do not.

In this work, we present a high-level computational model of IT-mediated crowds for collective intelligence. We introduce the Crowd Capital perspective as an organizational-level model of collective intelligence generation from IT-mediated crowds, and specify a computational system including agents, forms of IT, and organizational knowledge.

Can Crowdsourcing be used for policy? Previous work posits that the three types of Crowdsourcing have different levels of potential usefulness when applied to the various stages of the policy cycle. In this paper, we build upon this exploratory work by categorizing the extant research with respect to Crowdsourcing for the policy cycle. Premised upon our analysis, we thereafter discuss the trends, highlight the gaps, and suggest some approaches to empirically validate the application of Crowdsourcing to the policy cycle.

Can non-experts Crowds perform as well as experts in the assessment of policy measures? To what degree does geographical location relevant to the policy context alter the performance of non-experts in the assessment of policy measures? This research in progress seeks to answer these questions by outlining experiments designed to replicate expert policy assessments with non-expert Crowds. We use a set of ninety-eight policy measures previously evaluated by experts, as our control condition, and conduct the experiments using two discrete sets of non-expert Crowds recruited from a Virtual Labor Market (VLM). We vary the composition of our non-expert Crowds along two conditions; participants recruited from a geographical location relevant to the policy context, and participants recruited at-large. In each case we recruit a sample of 100 participants for each Crowd. Each experiment is then repeated at the VLM with completely new participants in each group to assess the reliability of our results. We will present results on the performance of all four groups of non-experts jointly and severally in comparison to the expert assessments, and discuss the ramifications of our findings for the use of non-expert Crowds and VLM’s for policy design. Our experimental design applies climate change adaptation policy measures.

To understand the observed and potential implications of IT-mediated Crowds for Politics and Policy, our research seeks to outline the contours of this emerging domain. We build the first-known dataset of IT-mediated Crowd applications currently in use in the governance context. From this starting point, we seek to understand the type of actors implementing these endeavours and the nature of the resources that they are generating. Early results from our dataset of 133 such applications indicates that there are a diversity of actors to be found engaging IT-mediated Crowds in the governance context, and that said actors are generating a large variety of resources from these activities. We then introduce the Crowd Capital Perspective to structure our investigations in this domain.

Crowdsourcing is rapidly evolving and applied in situations where ideas, labour, opinion or expertise of large groups of people is used. Crowdsourcing is now used in various policy-making initiatives; however, this use has usually focused on open collaboration platforms and specific stages of the policy process, such as agenda-setting and policy evaluations. Other forms of crowdsourcing have been neglected in policy-making, with a few exceptions. This article examines crowdsourcing as a tool for policy-making and explores the nuances of the technology and its use and implications for different stages of the policy process. The article addresses questions surrounding the role of crowdsourcing and whether it can be considered as a policy tool or as a technological enabler and investigates the current trends and future directions of crowdsourcing.

For what purposes are crowds being implemented in health care? Which crowdsourcing methods are being used? This work begins to answer these questions by reporting the early results of a systematic literature review of 110 pieces of relevant research. The results of this exploratory research in progress reveals that collective intelligence outcomes are being generated in three broad categories of public health care; health promotion, health research, and health maintenance, using all three known forms of crowdsourcing. Stemming from this fundamental analysis, some potential implications of the research are discussed and useful future research is outlined.

In this work we seek to understand how differences in location affect participation outcomes in IT-mediated crowds. To do so, we operationalize Crowd Capital Theory with data from a popular international creative crowdsourcing site, to determine whether regional differences exist in crowdsourcing participation outcomes. We present the early results of our investigation from data encompassing 1,858,202 observations from 28,214 crowd members on 94 different projects in 2012. Using probit regressions to isolate geographic effects by continental region, we find significant variation across regions in crowdsourcing participation. In doing so, we contribute to the literature by illustrating that geography matters in respect to crowd participation. Further, our work illustrates an initial validation of Crowd Capital Theory as a useful theoretical model to guide empirical inquiry in the fast-growing domain of IT-mediated crowds.

Crowd Science 2018 is proud to welcome four impactful new works to the field this year, and in this short paper we provide a brief review of the new works in advance of their presentation at HICSS 51. We conclude this brief introduction with a look ahead to the Crowd Science 2019 minitrack, while illustrating numerous useful subjects for future research.

Crowdsourcing is rapidly evolving and applied in situations where ideas, labour, opinion or expertise of large groups of people are used. Crowdsourcing is now used in various policy making initiatives; however, this use has usually been focused on open collaboration platforms and specific stages of the policy process such as agenda-setting and policy evaluations. Moreover, other forms of crowdsourcing have been neglected in policy making with a few exceptions. This article examines crowdsourcing as a tool for policy making and explores the nuances of the technology and its use and implications for different stages of the policy process. The article addresses questions around the role of crowdsourcing and whether it can be considered as a policy tool or a technology enabler.

What are the similarities and differences between crowdsourcing and sharing economy? What factors influence their use in developing countries? In light of recent developments in the use of IT-mediated technologies, such as crowdsourcing and the sharing economy, this manuscript examines their similarities and differences, and the challenges regarding their effective use in developing countries. We first examine each individually and highlight different forms of each IT-mediated technology. Given that crowdsourcing and sharing economy share aspects such as the use of IT, a reliance on crowds, monetary exchange, and the use of reputation systems, we systematically compare the similarities and differences of different types of crowdsourcing with the sharing economy, thus addressing a gap in the current literature. Using this knowledge, we examine the different challenges faced by developing countries when using crowdsourcing and the sharing economy, and highlight the differences in the applicability of these IT-mediated technologies when faced with specific development issues.

Two parallel phenomena are gaining attention in human-computer interaction research: gamification and crowdsourcing. Because crowdsourcing's success depends on a mass of motivated crowdsourcees, crowdsourcing platforms have increasingly been imbued with motivational design features borrowed from games; a practice often called gamification. While the body of literature and knowledge of the phenomenon have begun to accumulate, we still lack a comprehensive and systematic understanding of conceptual foundations, knowledge of how gamification is used in crowdsourcing, and whether it is effective. We first provide a conceptual framework for gamified crowdsourcing systems in order to understand and conceptualize the key aspects of the phenomenon. The paper's main contributions are derived through a systematic literature review that investigates how gamification has been examined in different types of crowdsourcing in a variety of domains. This meticulous mapping, which focuses on all aspects in our framework, enables us to infer what kinds of gamification efforts are effective in different crowdsourcing approaches as well as to point to a number of research gaps and lay out future research directions for gamified crowdsourcing systems. Overall, the results indicate that gamification has been an effective approach for increasing crowdsourcing participation and the quality of the crowdsourced work; however, differences exist between different types of crowdsourcing: the research conducted in the context of crowdsourcing of homogenous tasks has most commonly used simple gamification implementations, such as points and leaderboards, whereas crowdsourcing implementations that seek diverse and creative contributions employ gamification with a richer set of mechanics.

Please read the attached PDF for our call for papers for the crowdsourcing mini track.

The Bitcoin digital currency appeared in 2009. Since this time, researchers and practitioners have looked “under the hood” of the open source Bitcoin currency, and discovered that Bitcoin’s “Blockchain” software architecture is useful for non-monetary purposes too. By coalescing the research and practice on Blockchains, this work begins to unpack Blockchains as a general phenomenon, therein, arguing that all Blockchain phenomena can be conceived as being comprised of transaction platforms and digital ledgers, and illustrating where public key encryption plays a differential role in facilitating these features of Blockchains.

IT-mediated crowds are being implemented for multifarious purposes, using multifarious techniques. In this minitrack we seek to coalesce a specific and enduring community of IS and IS-related researchers focused on the study of IT-mediated crowds as a phenomenon. Our aim is to harness, and thus focus, the currently very broad inter-disciplinary study of IT-mediated crowds within the IS discipline proper, to incite a sharing of results and a cross-pollination of ideas among researchers currently looking at IT-mediated crowds from IS, I-School, HCI, Computer Science, Marketing, Education, Natural Sciences, Communication, and Technology Innovation perspectives.