No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Personalized Task Recommendation in Crowdsourcing Information Systems – Current State of the Art
Abstract
Crowdsourcing information systems are socio-technical systems that provide informational products or services by harnessing the diverse potential of large groups of people via the Web. Interested individuals can contribute to such systems by selecting among a wide range of open tasks. Arguing that current approaches are suboptimal in terms of matching tasks and contributors’ individual interests and capabilities, this article advocates the introduction of personalized task recommendation mechanisms. We contribute to a conceptual foundation for the design of such mechanisms by conducting a systematic review of the corresponding academic literature. Based on the insights derived from this analysis, we identify a number of issues for future research. In particular, our findings highlight the need for more significant empirical results through large-scale online experiments, an improved dialogue with mainstream recommender systems research, and the integration of various sources of knowledge that exceed the boundaries of individual systems.
... For example, identifying and closing accounts that publish fake news can control the source of dissemination [5,6] . Managing the procedure by which fake news is obtained or disseminated assists with the containment of its dissemination [7,8] . More importantly, strategies for curbing fake news on the user end can effectively increase the possibility of limiting fake news at an early stage of dissemination. ...
... Curbing fake news in the process means detecting key nodes and disrupting the fake news dissemination network. Zhu et al. [8] proposed a heuristic greedy algorithm to block the interaction of users exposed to misinformation in online social networks, to minimize the total amount of misinformation interaction between users and to curb the spread of misinformation. Cheng et al. [16] proposed a deep learning framework inspired by network science to predict the central nodes of a misinformation network and remove them to curb the spread of misinformation. ...
Curbing the dissemination of fake news in social media has been a major issue in recent years. Previous studies have suggested that the general public can recognize fake news, showing the feasibility of applying crowd ratings to identify fake news. However, the effectiveness of crowd ratings for curbing the dissemination of fake news is uncertain. This study constructed an online experimental platform to simulate Sina Microblog and designed a crowd rating strategy to compare and validate the difference between the absence vs. the presence of crowd ratings, and crowd ratings vs. expert ratings, in curbing the dissemination of fake news. We found that the presence of crowd ratings inhibited users' dissemination of fake news compared to the absence of crowd ratings. Moreover, there was no significant difference between the suppression effects of crowd ratings and expert ratings, both of which were effective in curbing the dissemination of fake news. Crowd rating uses collective intelligence to intervene in users' perceptions and behaviors at the onset of fake news dissemination, providing a cost-effective and efficient solution to combat the spread of fake news on social media.
... Citizen Qualification: Clearly specify the skills, abilities, and expertise required from citizens for participation. This is the most important determinant of meaningful participation [102], [103], [104]. To upgrade the qualifications of citizens, crowd coaching should be used, where experienced participants may assist new participants. ...
... It was also observed that for the convenience of citizens, the task was divided and split into smaller tasks [97], [98], [99], [100], [101], making complex problemsolving easy. Furthermore, as a diverse pool of citizens participates, the government decides the citizens' qualifications [102], [103], [104] to participate in advance. This principle restrains the unwanted traffic of citizens from landing on crowdsourcing initiatives, which, if not controlled, results in a waste of time and resources of the government. ...
Crowdsourcing in governance has gained widespread popularity and has become a powerful tool to leverage the collective intelligence of citizens for the nation's social welfare. Despite the growing importance of transforming government–citizen relationships, the design aspect is highly unstructured and fragmented. Thus, an in-depth understanding of the notion is essential, given its expanding importance in practice. This article aims to identify and describe elements and design principles phase-wise to improve the government's potential to derive value from crowdsourcing initiatives for civic participation and collaborative governance. Synthesizing all elements and principles in a strategic crowdsourcing framework is a novel conceptual configuration that contributes to the overall understanding of designing crowdsourcing initiatives in the government sector. The article adopts a systematic literature review and a morphological analysis technique to identify and synthesize all the elements and principles of designing crowdsourcing initiatives. Three crowdsourcing initiatives of the government sector were investigated to evaluate the framework in a practical setting and comprehend its efficacy. Government agencies can ensure enhanced interactivity and bidirectionality by emphasizing the nuances of elements and design principles for executing crowdsourcing initiatives that create epistemic and economic value.
... Moreover, the archetypes proposed by Geiger and Schader [15] imply four potential business models. By distinguishing (1) whether contributions are heterogeneous or homogeneous and (2) whether value is derived from individual contributions or all contributions in their entirety, four systems are elaborated: crowd rating, creation, processing, and solving systems. ...
... Another relevant issue is task liquidity, which considers both task dissemination and distribution. Scholars have developed different strategies, models, and algorithms for task recommendation and distribution by considering and prioritizing multiple factors such as crowd expertise, interest, and reliability [15], [32], [33]. The common aim is to deliver the right tasks to the right individuals at the right time. ...
Nowadays, crowdsourcing has become a popular way of sourcing. As intermediaries that connect crowdsourcers and crowds, crowdsourcing platforms integrate state-of-the-art information technologies and specialized organizational functions to host and govern crowdsourcing projects. The extant literature on crowdsourcing has investigated numerous aspects of crowdsourcing platforms. However, a majority of studies are project-oriented and short-term focused. There is a lack of a holistic view of crowdsourcing platforms as enterprises with a developmental perspective. This study aims to address this issue by investigating business sustainability of crowdsourcing platforms. By considering temporal dimensions and multiple interpretations of business sustainability, a conceptual framework is proposed to investigate the sustainability of a crowdsourcing platform by analyzing the key business process, value co-creation, and business development, which is a major theoretical contribution of the study. A case study of LEGO Ideas is presented to illustrate the practical implementation of the proposed framework. Both theoretical and practical implications are discussed.
... With the rapid growth in the number of workers and tasks [22], it has become increasingly difficult for workers to select the right task. The task assignment for workers and tasks is often a time-consuming process [23,24]. The time cost by a worker to select a suitable task is even comparable to the time cost to complete a task [25]. ...
... The time cost by a worker to select a suitable task is even comparable to the time cost to complete a task [25]. That is why a personalized task recommendation mechanism is particularly important [23]. How to recommend suitable tasks for workers has become a research hotspot in current crowdsourcing systems. ...
With the development of the Internet of Things and the popularity of smart terminal devices, mobile crowdsourcing systems are receiving more and more attention. However, the information overload of crowdsourcing platforms makes workers face difficulties in task selection. This paper proposes a task recommendation model based on the prediction of workers’ mobile trajectories. A recurrent neural network is used to obtain the movement pattern of workers and predict the next destination. In addition, an attention mechanism is added to the task recommendation model in order to capture records that are similar to candidate tasks and to obtain task selection preferences. Finally, we conduct experiments on two real datasets, Foursquare and AMT (Amazon Mechanical Turk), to verify the effectiveness of the proposed recommendation model.
... Regarding the second category, tasks, an incentive that has started to attract the interest is personalised task recommendation (providing tasks based on users' interest and capabilities) (Geiger and Schader, 2014;Yuen et al., 2015;Aldhahri et al., 2015). The reason is that recommendation systems have been highly successful in many other areas, such as product recommendation (recommend products based customers' interest) or content personalisation (show appropriate contents to each user based on their interest) (Geiger and Schader, 2014). ...
... Regarding the second category, tasks, an incentive that has started to attract the interest is personalised task recommendation (providing tasks based on users' interest and capabilities) (Geiger and Schader, 2014;Yuen et al., 2015;Aldhahri et al., 2015). The reason is that recommendation systems have been highly successful in many other areas, such as product recommendation (recommend products based customers' interest) or content personalisation (show appropriate contents to each user based on their interest) (Geiger and Schader, 2014). And if this approach is also successfully applied in crowdsourcing, it not only helps users find tasks faster and keep their motivations (because they do not take much time to find and choose the right tasks) but also helps requesters receive results with better quality (because the tasks are performed by the right users) and quicker. ...
Crowdsourcing is emerging as an efficient approach to solve a wide variety of problems by engaging a large number of Internet users from many places in the world. However, the success of these systems relies critically on motivating the crowd to contribute, especially in microtask crowdsourcing contexts when the tasks are repetitive and easy for people to get bored. Given this, finding ways to efficiently incentivise participants in crowdsourcing projects in general and microtask crowdsourcing projects in particular is a major open challenge. Also, although there are numerous ways to incentivise participants in microtask crowdsourcing projects, the effectiveness of the incentives is likely to be different in different projects based on specific characteristics of those projects. Therefore, in a particular crowdsourcing project, a practical way to address the incentive problem is to choose a certain number of candidate incentives, then have a good strategy to select the most effective incentive at run time so as to maximise the cumulitive utility of the requesters within a given budget and time limit. We refer to this as the incentive selection problem (ISP).
We present algorithms (HAIS and BOIS) to deal with the ISP by considering all characteristics of the problem. Specically, the algorithms make use of limited financial and time budgets to have a good exploration-exploitation balance. Also, they consider the group-based nature of the incentives (i.e., sampling two incentives with different group size yields two different number of samples) so as to make a good decision on how many times each incentive will be sampled at each time. By conducting extensive simulations, we show that our algorithms outperform state-of-the-art approaches in most cases. Also from the results of the simulations, practical usage of the two algorithms is discussed.
... resolving scientific problems) (Bayus, 2013;Natalicchio et al., 2017). According to Geiger and Schader (2014), collaborative crowdsourcing platforms can be further divided into crowdrating and crowdcreating platforms. On crowdrating platforms, solvers perform collective assessments or predictions by providing large homogeneous 'votes' or 'ratings' (Surowiecki, 2005). ...
... For instance, on the NASA clickworkers site, the clicks/votes generated by a large crowd of users were collected to detect craters on asteroids (Kanefsky et al., 2001). In contrast, on crowdcreating platforms, solvers collectively participate in creating more comprehensive and complex artefacts by making heterogeneous contributions (Geiger and Schader, 2014). Some crowdcreating platforms are leisure-orientated and help users create entertaining content and share it with their peers (B. ...
In recent years, gamification mechanics have been extensively adopted by crowdsourcing platforms to improve solvers’ participation and user experience. However, although gamified crowdsourcing on competitive platforms has frequently been investigated, gamified collaborative crowdsourcing platforms are poorly understood, especially platforms where solvers cooperatively contribute knowledge. It remains unclear how solvers’ intrinsic and extrinsic motivations mediate the relationship between gamification mechanics and solvers’ knowledge contribution. Based on self-determination theory and related literature, this study theorises the mediating roles of three intrinsic motivations (self-esteem, competence enhancement, and a sense of virtual community) and extrinsic motivations in the relationship between three typical gamification mechanics (immersion, social, and achievement) and solvers’ knowledge contribution. It then tests the hypotheses using survey data from 386 solvers from a large collaborative knowledge crowdsourcing platform. The empirical results show that self-esteem and competence enhancement positively mediate the impact of gamification mechanics on knowledge contribution, whereas extrinsic motivation negatively mediates this impact. The theoretical contributions and practical implications of this study are discussed.
... For this reason, unlike their simpler cousins, macrotasks usually require specific skills and knowledge to be accomplished (Schmitz and Lykourentzou, 2018). There is a wide variety of problems and types of tasks that today are being addressed with the help of macrotask crowdsourcing (Wang et al., 2021;Gimpel et al., 2023;Kohler and Chesbrough, 2019;Mcgahan et al., 2021;Geiger and Schader, 2014). ...
This paper investigates the shift in crowdsourcing towards self-managed enterprises of crowdworkers (SMECs), diverging from traditional platform-controlled models. It reviews the literature to understand the foundational aspects of this shift, focusing on identifying key factors that may explain the rise of SMECs, particularly concerning power dynamics and tensions between Online Labor Platforms (OLPs) and crowdworkers. The study aims to guide future research and inform policy and platform development, emphasizing the importance of fair labor practices in this evolving landscape.
... • генерація ідей та рішень, збір пропозицій; • винагороду учасників [25]. Як можна помітити, ця система зосереджена виключно на механізмах та організаційній структурі. ...
The article is devoted to the peculiarities of using crowd technologies in public institutions under conditions of military law in Ukraine. An analysis is made of the definition of crowd technologies in domestic and foreign literature. Criteria and indicators of crowd technologies are determined, including: a common goal as a factor of participants’ identity; absence of legal contracts and agreements; operational horizontal communication; a small amount of resources spent; non-financial incentives and motivation for attracting new participants; generation of decisions and ideas, obtaining new knowledge; effective implementation on the Internet; unlimited number of participants. The systematic structure of crowd technological projects and the advantages of crowd technologies in public administration compared to traditional administrative levers are systematized. An overview of crowd technology typologies at the level of governance institutions is provided, including their time of implementation, initiator status, and more. The reasons for the rapid development of crowd technologies in the conditions of a state of war have been identified, including: a high level of civic activity and responsibility of the population due to the efforts to counter Russian aggression; a high degree of development of civil society, especially the volunteer sector in the period preceding the full-scale Russian invasion; a culture of openness cultivated by the authorities following the Revolution of Dignity in 2014; a high level of trust in the military organization of the state and the overall legitimacy of the government, which became the center of unity for society; the presence of an information and network society actively introduced in Ukraine as a result of active policies of decentralization and digitization in previous years; a high level of development of the Internet and information and communication technologies, including as a result of the COVID-19 pandemic. The peculiarities and examples of the application of crowd technologies in conditions of martial law in Ukraine have been investigated. Various types of crowdfunding, based on voluntary contributions, including crowdsourcing, crowddonating, crowdlending, and crowdinvesting, are classified as financial crowd technologies. Crowdstaffing, crowdrecruiting, crowdtraining, crowdhunting, and crowdassessment are classified as personnel crowd technologies responsible for personnel recruitment. The focus is on crowd technologies that optimize the activities of public authorities and organizations and implement innovations. The following are studied: crowdmarketing, crowdcomputing, crowdstorming, crowdforesight, crowdtesting, crowdcrowdcreation, crowdwiki, crowdactive, crowdfixing, crowdsearching, crowdmapping, crowdsolving, and crowdvoting. Conditions and recommendations for using the opportunities of crowd technologies in the governance system, including in conditions of martial law, are identified, including decentralization of governance, e-governance, and informatization. The conditions and recommendations for utilizing the potential of crowd technologies in the governance system have been identified, including in the context of martial law.
... Difallah et al. [17] proposed a system that utilizes social networks to construct workers' preferences for worker-task matching. Geiger and Schader [18]investigated both task requesters and task keywords to estimate workers' task preferences in crowdsourcing systems. Zhu et al. [19] employed a Conditional Random Field to learn task characteristics from task descriptions and developers' characteristic distributions from their historical tasks. ...
The gig economy has facilitated the growth of customized services through digital platforms that connect consumers with service providers. However, the surge in service providers has led to a “cold-start problem", which limits the effectiveness of personalized task recommendation systems. To address this challenge, this paper purposed a personalized recommendation system for human-centric consumer services in the gig economy. It addresses the problem by using meta-learning to generate suitable preference embeddings for workers with limited bidding history, interests, and working competence. The system includes a competence module with self-attention and interest modules to capture workers’ personalized preferences. The model is evaluated on real-world datasets from Freelancer.com, and the results demonstrate that it outperforms state-of-the-art models in accurately recommending suitable personalized tasks to both new and existing workers with skill-evolving. The proposed system can reduce task completion times and improve task quality by ensuring that tasks are assigned to the most suitable workers.
... On the other hand, when it comes to the classification based on the type of task to be carried out, crowdsourcing can be divided into four general categories (Geiger, Fielt, Rosemann, & Schader, 2012;Geiger & Schader, 2014): crowdsolving (solving the problem by the community), crowdcreation (co-creating complex products by the community based on a variety of user-generated content), crowdrating (collective evaluations or forecasting), and crowdprocessing (engaging the community in a large number of homogeneous activities in order to implement a specific project). From this perspective, crowdfunding should be classified under the last categorycrowdprocessing -as raising funds allows for the implementation of a specific business or social project. ...
This is the first book to focus on crowdfunding in sport. Crowdfunding is an important new financial instrument that is becoming more popular with sports organisations, and this book examines the research evidence for crowdfunding and considers how it might be successfully implemented.
Presenting international cases and data, including from European football, the book explains how crowdfunding campaigns have to be fully integrated with strategic marketing plans and require a solid understanding of the needs and motivations of potential investors, consumers, and fans. The book sets out a theoretical framework for applying strategic marketing in the context of crowdfunding in sports clubs, introduces the key characteristics of the sports crowdfunding market and funders’ behaviours in the crowdfunding campaigns of sports clubs, examines the market segments of the campaigns’ funders, and presents recommendations for developing marketing-mix programs to target them.
This is important reading for any researcher, advanced student, or practitioner with an interest in sport business, sport marketing, sport finance, consumer behaviour in sport, or entrepreneurship, innovation, or technology in sport.
... On the other hand, when it comes to the classification based on the type of task to be carried out, crowdsourcing can be divided into four general categories (Geiger, Fielt, Rosemann, & Schader, 2012;Geiger & Schader, 2014): crowdsolving (solving the problem by the community), crowdcreation (co-creating complex products by the community based on a variety of user-generated content), crowdrating (collective evaluations or forecasting), and crowdprocessing (engaging the community in a large number of homogeneous activities in order to implement a specific project). From this perspective, crowdfunding should be classified under the last categorycrowdprocessing -as raising funds allows for the implementation of a specific business or social project. ...
The purpose of this chapter is to provide a comprehensive description of crowdfunding and to organise the current knowledge on participant behaviour and marketing management decisions made by campaign initiators. This chapter answers questions such as what is crowdfunding and how does it differ from crowdsourcing? How has crowdfunding evolved and what are the various crowdfunding models and the criteria for their selection by initiators? Additionally, the chapter explores the motivations and decision-making criteria of campaign participants, the different segments of crowdfunding participants, the components of the campaign product, the pricing and distribution strategies, and the most effective marketing communication strategies for achieving campaign objectives. By reviewing existing research, this chapter aims to provide a concise summary of marketing decisions in the area of crowdfunding, making it useful for initiators of campaigns beyond the realm of sports-related activities.
... 19 In OHCs, the Causal effect of honorary titles on physicians' service volumes, rather than the e-consult accessibility on the patient assessments importance of online voting for sharing patient experience has been verified by extensive studies. 20,21 Patients label and vote on physicians' expertise, and then this information records the diversity of physician experience. Patients' vote volume reflected clinicians' online contribution and engagement, 22 which were investigated in studies of word-of-mouth 14 and experience sharing. ...
Objective
In online health communities (OHCs), patients often list their physicians’ expertise by user-generated tags based on their consulted diseases. These expertise tags play an essential role in recommending the match of physicians to future patients. However, few studies have examined the impact of the accessibility of e-consults on patient assessments using marking of the physicians’ expertise in OHCs. This study aims to investigate what are the patient assessments of the physicians’ expertise if they have e-consult accessibility.
Methods
Through a case–control study, this article examined the effect of e-consult accessibility on patient-generated tags of physician expertise in OHCs. With data collected from the Good Doctor website, the samples consisted of 9841 physicians from 1255 different hospitals widely distributed in China. The breadth of voted expertise (BE) is measured by the number of consulted disease-related labels marked by a physician's served patients (SP). The volume of votes (VV) is measured by the number of a physician's votes given by the SP. The degree of voted diversity (DD) is measured by the information entropy of each physician's service expertise (labeled and voted by patients). The data analysis of e-consult accessibility is conducted by estimating the average treatment effect on the DD of physicians’ expertise.
Results
For the BE, its mean was 7.305 for the case group of physicians with e-consults accessible (photo and text queries), while the mean was 9.465 for the control for physicians without e-consults. For the VV, its mean was 39.720 for the case group, while the mean was 84.565 for the control. For the DD, its mean on patient-generated tags was 2.103 for the case group, which is 0.413 lower than the control group.
Conclusion
The availability of e-consults increases the concentration on physician expertise in the patient-generated tags. e-Consults reinforce the increment of the already-received physician expertise (reflected in tags), reducing the tag information diversity.
... To address this issue, crowdworkers turn to the use of HIT automated scripts (henceforth referred to as catching scripts), such as Panda Crazy and MTurk Suite (Irani & Silberman, 2013;Ramirez, 2021;TurkerView mTurk Forum, 2021). These scripts allow crowdworkers to 'catch' HITs posted by specific job requesters (based on their ID number), with high ratings or based on the specificities of the HIT (Saito et al., 2019), whereby the requirements of the HIT fit their prior experience or personal interests (Dror et al., 2011;Geiger & Schader, 2014). In other words, these scripts support selective and automatic catching of HITs based on personal preferences. ...
Crowdworkers on platforms like Amazon Mechanical Turk face growing competition as a result of the global excess supply of digital labour. As a result, many crowdworkers turn to automated scripts, which help them locate better tasks faster and to boost their earnings. However, to date, it is not clear whether and to what extent the use of such scripts influence the opportunities for those crowdworkers who do not use them. This an important aspect that warrants further exploration because it can have negative implications for the health of crowdwork platforms. In this study, we use Discrete Event Simulation to identify and quantify the unintended consequences of the excessive use of automated scripts. Our findings show that, while the use of scripts allows some crowdworkers to identify and accept far more tasks than others, in the long run, this behaviour results in their competence persistence and reputational persistence and progressively to detrimental impacts for those workers who do not use scripts, and who may ultimately be forced to exit the platform. As a result, automated scripts have negative consequences, whereby their excessive use leads to a tragedy of the commons for all platform stakeholders, including the crowdworkers, the job requesters and the platform itself.
... Wikidata is an open, multilingual, freely available knowledge graph edited by a global community of volunteers from diverse backgrounds and experiences (Piscopo et al, 2017b;Geiger and Schader, 2014). There are currently over 300k registered Wikidata editors who can contribute to the KG with or without creating a user account. ...
Wikidata is an open knowledge graph created, managed, and maintained collaboratively by a global community of volunteers. As it continues to grow, it faces substantial editor engagement challenges, including acquiring new editors to tackle an increasing workload and retaining existing editors. Experiences from other online communities and peer-production systems, including Wikipedia, suggest that recommending tasks to editors could help with both. Our aim with this paper is to elicit the user requirements for a Wikidata recommendations system. We conduct a mixed-methods study with a thematic analysis of in-depth interviews with 31 Wikidata editors and three Wikimedia managers, complemented by a quantitative analysis of edit records of 3,740 Wikidata editors. The insights gained from the study help us outline design requirements for the Wikidata recommender system. We conclude with a discussion of the implications of this work and directions for future work.
... Organizations are looking for vendors, consultants, and researchers who can assist them in this transformation. This is evident in the academic research which is now exploring sourcing topics such as crowdsourcing (Blohm et al., 2013;Geiger & Schader, 2014), platform ecosystems (Constantinides et al., 2018;Foerderer et al., 2018;Ghazawneh & Henfridsson, 2013;Huber et al., 2017;Schmeiss et al., 2019;Tiwana, 2002), cloud computing (Venters & Whitley 2012;Schneider & Sunyaev, 2016;Yinghui et al., 2018), service innovation (Barrett et al., 2015;Lusch & Nambisan, 2015), service automation (robotic process automation-RPA) Rutschi & Dibbern, 2020;, impact sourcing (Heeks, 2013;Sandeep & Ravishankar, 2018); artificial intelligence/machine learning (Davenport & Ronanki, 2018), process mining/analytics (Fogarty & Bell, 2014), internet of things (Dijkman et al., 2015), and blockchain (cf. . ...
In this chapter, I attempt to document the early days of Information Technology Outsourcing, starting with the initial EDS facilities management contracts, which then led to the notion of IT outsourcing and its proliferation. I will present the history of the mega-deals, and some of the ideas, lessons, and rationale, which drove early outsourcing. By understanding the early history of IT outsourcing, it better positions us to appreciate the evolution and challenges facing today’s new forms of outsourcing.
... Deep learning algorithms are powerful because they can learn and deal with complex problems such as human Mobile Information Systems beings, analyze and calculate linear or nonlinear feature sequences from multiple dimensions in the face of complex scale data, and automatically learn features that meet users' needs from massive data [20]. Deep learning techniques can not only discover the hidden potential features of user behavior records, but also capture the interaction features of user-user, user-item, and item-item nonlinear relationships, which brings more opportunities for system performance improvement and can overcome some obstacles encountered in traditional recommendation techniques to achieve more accurate recommendations. ...
Big Data is the most popular concept in this era, which is the massive amount of information and related technology generated by the information explosion in the era of “Internet+.” Big Data is the most popular concept of our time. With the most advanced technology to collect, analyze, organize, and store data, Big Data can effectively handle all kinds of complex information. Because of this, big data is widely favored by all walks of life. In China’s sports industry, the use of big data has become mature and has shown its unique advantages. With the development of campus soccer in China in the past decade, how to use big data to promote the sustainable development of campus soccer in China has become a key issue for sports workers to consider today. Based on the above background, this paper proposes a system combining data mining and personalized data recommendation to collect and analyze the information of campus soccer to promote the sustainable development of campus soccer. First, we propose a data mining method based on deep learning data mining network model combined with migration learning to address the data mining problem. The method uses the knowledge of historical model parameters and applies them to new tasks, thus solving the problem of network training when samples are lacking and improving data utilization and data mining effects. Then, for the data recommendation problem, a new deep learning method is proposed, which performs effective intelligent recommendation by pretraining. In the initial phase, the corresponding low-dimensional embedding vectors are learned, which capture information reflecting the relevance of students to soccer sports. During the prediction phase, a feed-forward neural network is used to model the interaction of student and soccer sport information, where the corresponding pretrained representative vectors are used as inputs to the neural network. Finally, it is experimentally verified that the data mining method proposed in this paper can effectively improve the data mining performance and efficiency, and the proposed data recommendation method possesses better accuracy than the traditional methods. The use of this system can effectively collect and analyze campus soccer information, which helps to develop campus soccer and promote the sustainable development of campus soccer.
... The next aspect to be mentioned is quality assurance. Ideally, this can be done in such a way those employees within the organisation carry out at least a first review of the solutions (Geiger and Schader, 2014). Via the social feedback processes mentioned above, the feedback is then passed on to the contractor in order make possible improvements. ...
... The next aspect to be mentioned is quality assurance. Ideally, this can be done in such a way those employees within the organisation carry out at least a first review of the solutions (Geiger and Schader, 2014). Via the social feedback processes mentioned above, the feedback is then passed on to the contractor in order make possible improvements. ...
... Pour les organisateurs, elles sont « un moyen d'obtenir des gains de productivité inégalés en exploitant les actifs sous-utilisés et en générant par l'agrégation des informations un apprentissage exponentiel des comportements et des habitudes » (Benavent, 2016, p. 21). Dans le domaine du crowdsourcing, elles renvoient à la pratique d'externalisation de tâches, habituellement réservées aux employés d'une firme, vers un large groupe d'individus extérieurs à l'entreprise sous la forme d'un appel ouvert à la participation (Howe, 2006 ;Geiger et Schader, 2014). Leur développement a été très largement facilité par les technologies de l'information et de la communication (Cardon, 2006), en permettant de satisfaire le principe de l'appel ouvert et ainsi, de capitaliser sur les compétences, les expériences et la créativité de groupes d'individus hétérogènes (Renault, 2014a). ...
Les plateformes de crowdsourcing d’idées favorisent le travail de co-création. Pour les internautes, elles se révèlent comme des espaces d’expression reflétant leur pouvoir d’agir. De manière paradoxale, elles peuvent être considérées comme des espaces de mise au travail, la pu-blicisation de l’idée impliquant un transfert de propriété au profit de l’entité organisatrice. L’objectif de cette recherche, basée sur l’analyse empirique d’une plateforme, est de mieux comprendre le système des valeurs et les arrangements mis en œuvre par les acteurs pour justifier leurs pratiques. En nous appuyant sur le modèle conventionnaliste des « Economies de la grandeur » de Boltanski et Thevenot, nous mettons en avant la juxtaposition de différentes cités. Si le principe de rivalité constitue un puissant levier de captation des internautes, le registre de justification de la cité par projets s’impose comme modèle de référence chez les acteurs les plus centraux.
... Latoza et al. (Latoza & Hoek, 2015) also emphasized on the matching of workers with their expertise and knowledge and to get maximum benefit from the CSD worker is an issue. Similar is the case is discussed in the (Geiger & Schader, 2014;Gilal, Jaafar, Omar, Basri, & Din, 2016;Gilal, Jaafar, Omar, Basri, & Waqas, 2016;Gilal, Omar, & Sharif, 2013;Tunio et al., 2017) studies that while keeping extrinsic and intrinsic choice of CSD workers self-identification principle for individual contributors to select those tasks which are the best match with their psychological preferences (i.e., personality). Psychological is an important factor to compliance with the choice and individual capabilities with the respective task requirements. ...
An open call format of crowdsourcing software development (CSD) is harnessing potential, diverse, and unlimited people. But, several thousand solutions are being submitted at platform against each call. To select and match the submitted task with the appropriate worker and vice versa is still a complicated problem. Focusing the issue, this study proposes a task assignment algorithm (TAA) that will behave as an intermediate facilitator (at platform) between task (from requester) and solution (from worker). The algorithm will divide the tasks' list based on the developer's personality. In this way, we can save the time of both developers and platform by reducing the searching time.
... Crowdfunding is a form of crowdsourcing, which involves using the vast potential of contributors to obtain products or services, often time via the internet (Geiger & Schader, 2014). It is inspired by microfinance, social fundraising crowdsourcing (Morduch, 1999;Poetz & Schreier, 2012). ...
The European Union’s regional and interregional policy places the promotion of entrepreneurship and innovation (E&I) high in the agenda, anticipating that it may boost sustainable growth and social welfare. In this respect, the present volume delves into E&I issues that are important to Western Greece and Apulia (Puglia), by bringing together statistics, models, analyses, reviews, practical experiences, tools, and ideas, which, understandably, may be of interest or use to other parts of the world as well.
... Crowdsourcing platforms range in different disciplines such as entertainment, healthcare. According to Geiger and Schader (2014), crowdsourcing platforms have four archetypes (crowd processing, crowd solving, crowd creating, and crowd rating) based on a 2 3 2 matrix that differentiates value between value originating from contributors and contributors. Crowdsourcing in gaming falls into the category of crowd creating as it allows for heterogeneous contributions from Pok emon Go gamers. ...
Purpose
With the increasing popularity of online games like Pokémon Go, a new wave of crowdsourcing communities have emerged, allowing gamers to collaborate, communicate and share useful game-related information. This paper aims to examine the factors that influence gamers' crowdsourcing behaviour.
Design/methodology/approach
A conceptual framework is developed that combines the DeLone & McLean model, self-determination theory, and different levels of engagement behaviour. The online survey collected 371 responses that were analysed using Covariance Based Structural Equation Modelling (CB-SEM).
Findings
The results show that extrinsic and intrinsic motivation positively influenced gamers' crowdsourcing engagement intention. System quality and information quality were also confirmed to be positively associated with gamers' crowdsourcing engagement intention. Furthermore, crowdsourcing engagement intention was found to be positively associated with crowdsourcing content consumption, contribution, and creation.
Practical implications
The findings of this study are useful for the owners of Pokémon Go and other gaming-related crowdsourcing platforms in devising tailored strategies to increase the crowdsourcing engagement of gamers.
Originality/value
This study provides the first empirical evidence of factors motivating online gamers' crowdsourcing intention. This study also presents novel insight into online gamers' crowdsourcing intention by combining diverse theories which offer different perspectives and a more comprehensive understanding of the phenomenon. Contribution to the research on the intention-behaviour gap by modelling three behavioural outcomes (content creation, contribution, and consumption behaviour) of crowdsourcing engagement intention, is another important contribution of this study.
... The general approach of these crowdsourcing efforts is to focus on what to ask each contributor. Specifically, from a large set of possible tasks, many systems formalize an approach to route or recommend tasks to specific contributors [55,121,2,47]. Unfortunately, many of these volunteer efforts are restricted to labels for which contributions can be motivated, leaving incomplete any task that is uninteresting to contributors [155,69,66,196]. ...
Over the last decade, Computer Vision, the branch of Artificial Intelligence aimed at understanding the visual world, has evolved from simply recognizing objects in images to describing pictures, answering questions about images, aiding robots maneuver around physical spaces and even generating novel visual content. As these tasks and applications have modernized, so too has the reliance on more data, either for model training or for evaluation. In this chapter, we demonstrate that novel interaction strategies can enable new forms of data collection and evaluation for Computer Vision. First, we present a crowdsourcing interface for speeding up paid data collection by an order of magnitude, feeding the data-hungry nature of modern vision models. Second, we explore a method to increase volunteer contributions using automated social interventions. Third, we develop a system to ensure human evaluation of generative vision models are reliable, affordable and grounded in psychophysics theory. We conclude with future opportunities for Human-Computer Interaction to aid Computer Vision.
... To complement the above search methods, some researchers propose task recommendation algorithms [26][27][28][29][30] based on worker characteristics and task characteristics, in order to provide workers with more appropriate tasks to choose from. Ambati et al. [28], based on the historical interactions between workers and tasks, built a preference model for workers and learned workers' preferences through "Bag-of-Words Approach" and "Classification Based Approach," so as to recommend tasks that might be of interest to workers. ...
Crowdsourcing provides a distributed method to solve the tasks that are difficult to complete using computers and require the wisdom of human beings. Due to its fast and inexpensive nature, crowdsourcing is widely used to collect metadata and data annotation in many fields, such as information retrieval, machine learning, recommendation system, and natural language processing. Crowdsourcing helps enable the collection of rich and large-scale data, which promotes the development of researches driven by data. In recent years, a large amount of effort has been spent on crowdsourcing in data collection, to address the challenges, including quality control, cost control, efficiency, and privacy protection. In this paper, we introduce the concept and workflow of crowdsourcing data collection. Furthermore, we review the key research topics and related technologies in its workflow, including task design, task-worker matching, response aggregation, incentive mechanism, and privacy protection. Then, the limitations of the existing work are discussed, and the future development directions are identified.
Građanska znanost, u širem smislu, praksa je javne participacije i suradnje u znanstvenom istraživanju s ciljem povećanja znanstveno utemeljenog znanja. Iako je građanska znanost postojala stoljećima, termin „građanska znanost” skovan je 1990-ih i od tada dobiva na popularnosti. Izrastao iz područja sudjelovanja „građana amatera” u znanstvenim istraživanjima u nizu različitih disciplina te na temeljima primjene rada mnoštva (engl. crowdsourcing), danas je koncept građanske znanosti pozicioniran unutar šireg područja otvorene znanosti te je građanska znanost prepoznata kao utemeljen i validan pristup znanstvenim istraživanjima. Postoji više područja na kojima građanska znanost donosi brojne prednosti za sudionike i istraživače. Međutim, primjena pristupa građanske znanosti ima i svoja ograničenja i izazove, uključujući osiguravanje kvalitete podataka, održavanje interesa i uključenosti građana te često, skepticizam znanstvene zajednice i nedostatak uspostavljenih potpornih struktura za financiranje i provođenje projekata. Kako bismo pružili čitateljima jasnoću, razumijevanje i orijentaciju u ovom dinamičnom području, sastavili smo ovu knjigu kao sveobuhvatan vodič kroz građansku znanost – od njezinih povijesnih korijena do suvremenih primjena i budućih perspektiva. Glavni cilj ove knjige obrazovati je i osnažiti istraživače, građane i voditelje radionica i programa građanske znanosti pružajući uvide u teorijske osnove, metode primjene i strategije za evaluaciju. Kroz detaljnu analizu ova knjiga pokušava razjasniti što građanska znanost zaista predstavlja, kako se razvijala tijekom vremena i kako se danas primjenjuje. Poseban naglasak stavljen je na važnost građanske znanosti u društvu kroz prikaz primjera njezinih primjena u humanističkim i društvenim istraživanjima u kojima se može olakšati prikupljanje podataka, potaknuti šira uključenost zajednice i poticati bolje razumijevanje društvenih fenomena. Svaka cjelina ove knjige konstruirana je tako da čitatelj može steći sveobuhvatno razumijevanje građanske znanosti – od njezinih teorijskih temelja do praktičnih smjernica. Knjiga je dizajnirana tako da potakne daljnje istraživanje i raspravu o građanskoj znanosti doprinoseći njezinu kontinuiranom razvoju i implementaciji. Knjiga je koncipirana kroz šest komplementarnih poglavlja, polazeći od šireg povijesnog i konceptualnog određenja područja građanske znanosti pa do specifičnih odrednica primjene građanske znanosti u kontekstu društveno-humanističkih istraživanja. Knjiga je nastala u okviru projekta Građanska znanost u društvenim i humanističkim istraživanjima: teorijski okvir i smjernice za primjenu koji je podržan sredstvima Filozofskog fakulteta Sveučilišta u Zagrebu.
Online labor markets have gained significant importance in recent years, drawing considerable attention in academia and practice. These platforms enable workers worldwide to sell their labor services to a global pool of clients. However, the challenge lies in motivating workers effectively to enhance their productivity. To address this issue, we employ the self-determination theory and present a model that elucidates motivation's impact on productivity across various payment schemes. Additionally, we leverage the psychological trait theory and its suggested taxonomy to explore how compensation policies in online labor markets affect incentives differently based on individual differences. Our experiment tests predictions from a formal labor supply and productivity model for workers with varying compensation levels. The results indicate that intrinsic workers exhibit higher productivity when bonus rewards are introduced. Furthermore, our study confirms the presence of heterogeneous personality effects, emphasizing that increased worker productivity is primarily associated with conscientiousness and agreeableness traits. These findings illuminate the intricate mechanisms governing worker motivation and engagement in paid crowdsourcing environments. They provide valuable theoretical and managerial insights for researchers and crowdsourcing practitioners aiming to enhance worker productivity in online tasks. ARTICLE HISTORY
During the COVID-19 outbreak, crowdsourcing-based context-aware recommender systems (CARS) which capture the real-time context in a contactless manner played an important role in the "new normal". This study investigates whether this approach effectively supports users' decisions during epidemics and how different game designs affect users performing crowdsourcing tasks. This study developed a crowdsourcing-based CARS focusing on restaurant recommendations. We used four conditions (control, self-competitive, social-competitive, and mixed gamification) and conducted a two-week field study involving 68 users. The system provided recommendations based on real-time contexts including restaurants' epidemic status, allowing users to identify suitable restaurants to visit during COVID-19. The result demonstrates the feasibility of crowdsourcing to collect real-time information for recommendations during COVID-19 and reveals that a mixed competitive game design encourages both high- and low-performance users to engage more and that a game design with self-competitive elements motivates users to take on a wider variety of tasks. These findings inform the design of restaurant recommender systems in an epidemic context and serve as a comparison of incentive mechanisms for gamification of self-competition and competition with others.
Over the past two decades, crowdsourcing activities have expanded considerably. More recently, the COVID-19 pandemic has radically changed the way people live and work, and the way organizations do business. So far, not many studies have analyzed if and to what extent trustworthiness can influence the admiration to practice crowdsourcing and could reward financial benefits in the COVID-19 period. Against this background, in this article, the aim is to investigate the influence of crowdfunding trustworthiness and financial rewards on the success of crowdsourcing activities. The analysis is made more complete by including technology leadership support as a moderator. With the help of the existing literature and theories, a research model has been developed conceptually, which was later validated using the partial least square-structural equation modeling technique on a sample of 319 responses from participants based in Europe and Asia. The study found that lucidity, gamification, exposure, and coordination along with financial rewards positively influence admiration for crowdsourcing, which, in turn, positively impacts successful crowdsourcing practices in the COVID-19 period. The study also shows that there is a significant moderating impact of technological leadership support on successful crowdsourcing practices in the COVID-19 period.
Purpose
This study examines how contributors with different achievement goals participate under the influence of two common motivators/demotivators on crowdsourcing platforms, namely system design features and task nature.
Design/methodology/approach
A free simulation experiment was conducted among undergraduate students with the use of a crowdsourcing platform for two weeks.
Findings
The results indicate that contributors with a strong performance-approach goal get better scores and participate in more crowdsourcing tasks. Contributors with a strong mastery-avoidance goal participate in fewer heterogeneous tasks.
Research limitations/implications
Contributors with different achievement goals participate in crowdsourcing tasks to different extents under the influence of the two motivators/demotivators. The inclusion of the approach-avoidance dimension in the performance-mastery dichotomy enables demonstrating the influence of motivators/demotivators more specifically. This article highlights differentiation between the quality and the quantity of heterogeneous crowdsourcing tasks.
Practical implications
Management is advised to approach performance-approach people if a leaderboard and a point system are incorporated into their crowdsourcing platforms. Also, management should avoid offering heterogeneous tasks to mastery-avoidance contributors. System developers should take users' motivational goals into consideration when designing the motivators in their systems.
Originality/value
The study sheds light on habitual achievement goals, which are relatively stable in comparison to contributors' motives and states. The relationships between achievement goals and motivators/demotivators are more persistent across time. This study informs system designers' decisions to include appropriate motivators for sustained contributor participation.
This chapter provides a comprehensive overview of the phenomenon of review bomb, which occurs when an abnormally large amount of information is submitted to a rating system in a very short period of time by an overtly anonymous mass of accounts, with the overall goal of sabotaging the system's proper functioning. Because review bombs are frequently outbursts of social distress from gaming communities, gamification theories have proven useful for understanding the behavioral traits and conflict dynamics associated with such a phenomenon. A prominent case is analysed quantitatively. The methodology is discussed and proposed as a generalized framework for descriptive quantification of review bombs. As a result of the study, considerations for technological improvements in the collection of rating data in systems are proposed too.
Crowdsourcing recommendation is necessary to surmount information overload and assist workers to identify tasks of interest efficiently. However, existing studies mainly concentrate on investigating workers’ qualifications or requirements, neglecting their fuzzy expectations and psychological behavior. Besides, workers can only passively accept the results and do not have any right to change them even if they are not satisfactory. As workers will pay different attention to task aspects, we regard the crowdsourcing recommendation as a multi-criteria decision-making (MCDM) problem. In this paper, we propose an MCDM-based method for automated and interactive crowdsourcing task recommendations while comprehensively considering workers’ fuzzy expectations and psychological behavior. Given a large volume of available tasks on a crowdsourcing platform, the requirement of massive human efforts in traditional MCDM is not applicable in such situation. Therefore, we formulate a set of quantitative task evaluation criteria for automated task assessments based on the attributes that have the most impact on workers’ participation behavior. Further, to capture workers’ fuzziness in expectations, an intuitionistic fuzzy 2-tuple linguistic (IF2L) term is introduced, which are further leveraged as reference points to evaluate their “gains” and “losses” based on the prospect theory. Finally, the D-S evidence theory is adopted to gather workers’ uncertain prospect values. A case of ZBJ.COM, a popular crowdsourcing platform in China, shows that the proposed method is efficient and effective in recommending suitable and personalized tasks to workers.
Mobile crowdsourcing is rapidly boosting the Internet of Things revolution. Its natural development leads to an adaptation to various real-world scenarios, thus imposing a need for wide generality on data-processing and task-assigning methods. We consider the task assignment problem in mobile crowdsourcing while taking into consideration the following: (i) we assume that additional information is available for both tasks and workers, such as location, device parameters, or task parameters, and make use of such information; (ii) as an important consequence of the worker-location factor, we assume that some workers may not be available for selection at given times; (iii) the workers' characteristics may change over time. To solve the task assignment problem in this setting, we propose
Adaptive Optimistic Matching for Mobile Crowdsourcing
(AOM-MC), an online learning algorithm that incurs
regret in
T
rounds, for any
, under mild continuity assumptions. Here,
is a notion of dimensionality which captures the structure of the problem. We also present extensive simulations that illustrate the advantage of adaptive discretization when compared with uniform discretization, and a time- and location-dependent crowdsourcing simulation using a real-world dataset, clearly demonstrating our algorithm's superiority to the current state-of-the-art and baseline algorithms.
Crowdsourcing, as a crowd-centered approach, is becoming increasingly popular for organizations to conduct outsourcing, research and development (R&D), and marketing. The effectiveness of a crowdsourcing initiative, as manifested in specific outcomes, depends significantly on the salient characteristics of the configured crowd. This study aims to investigate which business purposes necessitate which crowds with which characteristics. Contributions of this study include: 1) defining a crowd in crowdsourcing by distinguishing the roles of individuals, 2) introducing and defining three crowd attributes to depict the salient characteristics of a crowd, and 3) proposing a typology of eight crowd configurations by combining high or low levels of the three crowd attributes and examining each crowd configuration to highlight the relationships between crowd attributes and crowdsourcing outcomes. Eight mini cases corresponding to the eight crowd configurations are presented to illustrate how crowd configurations were implemented in real-life situations. The theoretical and practical implications are discussed respectively.
RedEMC is an e-learning web platform that utilizes a hybrid recommender system to suggest learning resources and online courses to Latin American doctors and other health specialists in the context of Continuing Medical Education. Explicit and implicit feedback were collected in a period of six months to determine the usefulness of personalized recommendations through predictive models, using machine learning approaches and methods. The main contribution of this research is to show how to utilize the feedback given by students who received personalized recommendations in a real study case to create predictive machine learning models that assist organizations to analyze the efficiency of their recommender systems. Once predictive models are generated, educational institutions and companies could also utilize them to make strategic decisions regarding the accomplishment of their organizational goals.
KeywordsRecommender systemsMachine learninge-Learning
The crowdsourcing term was first defined as the outsourcing of tasks or work to a group of an unknown workers in an open call for the purpose of utilizing human intelligence. Crowdsourcing nowadays gained popularity in industries and academia around the globe. Various organization and communities takes interest in it because of its parallel and cost-efficient nature. While crowdsourcing, it is necessary to have a complete knowledge of crowdsourcing process. Crowdsourcing activity consist of three main actors. Current study briefly discussed the analyzed crowdsourcing actors crowd, client and platforms. Features of the crowd and role of the client have been discussed. Various crowdsourcing platforms have been analyzed and categorized in order to get deep knowledge of the tasks performed on it. The authors concludes their study by identifying various crowdsourcing applications in a variety of disciplines.
Learning idiomatic expressions is seen as one of the most challenging stages in second-language learning because of their unpredictable meaning. A similar situation holds for their identification within natural language processing applications such as machine translation and parsing. The lack of high-quality usage samples exacerbates this challenge not only for humans but also for artificial intelligence systems. This article introduces a gamified crowdsourcing approach for collecting language learning materials for idiomatic expressions; a messaging bot is designed as an asynchronous multiplayer game for native speakers who compete with each other while providing idiomatic and nonidiomatic usage examples and rating other players’ entries. As opposed to classical crowd-processing annotation efforts in the field, for the first time in the literature, a crowd-creating & crowd-rating approach is implemented and tested for idiom corpora construction. The approach is language-independent and evaluated on two languages in comparison to traditional data preparation techniques in the field. The reaction of the crowd is monitored under different motivational means (namely, gamification affordances and monetary rewards). The results reveal that the proposed approach is powerful in collecting the targeted materials, and although being an explicit crowdsourcing approach, it is found entertaining and useful by the crowd. The approach has been shown to have the potential to speed up the construction of idiom corpora for different natural languages to be used as second-language learning material, training data for supervised idiom identification systems, or samples for lexicographic studies.
This paper identifies antecedents to frustration arising in micro-task crowd work (MTCW) and measures their impact on a frustrating work experience. Surprisingly, research to date has said little on frustration in MTCW, despite it being a central challenge to working persistently in these increasingly important socio-technical work environments. Guided by an experiential computing perspective, we applied a mixed-method approach to reveal how the interplay between platforms and
other workers’ behavior shapes frustration experienced in MTCW. We present qualitative and quantitative findings on frustration in virtual MTCW. Based on 96 narratives, we qualitatively identify three dominant antecedents to frustration: technical problems in conducting tasks, platform competition and, the inability to disconnect from work. We subsequently conducted a quantitative two wave survey (N=181), to test the factors’ impacts on MTCW frustration. The results confirm the three factors and additionally reveal that autonomy largely reduces frustration. However, autonomy also reinforced frustration arising from the inability to disconnect from work. Thus, workers’ inability to
disconnect from work can drive excessive availability that lures them into taking on additional tasks. Thereby, workers experience additional frustration and underutilize the benefits of autonomy in crowd
work. Our manuscript offers three central contributions: First, we identify frustration as pivotal in the experience of MTCW and its antecedents. Second, we shed light on the heterogenous role of autonomy in relation to MTCW frustration. Finally, we link the findings on frustrating MTCW experience to experiential computing.
Over the last decade, Computer Vision, the branch of Artificial Intelligence aimed at understanding the visual world, has evolved from simply recognizing objects in images to describing pictures, answering questions about images, aiding robots maneuver around physical spaces, and even generating novel visual content. As these tasks and applications have modernized, so too has the reliance on more data, either for model training or for evaluation. In this chapter, we demonstrate that novel interaction strategies can enable new forms of data collection and evaluation for Computer Vision. First, we present a crowdsourcing interface for speeding up paid data collection by an order of magnitude, feeding the data-hungry nature of modern vision models. Second, we explore a method to increase volunteer contributions using automated social interventions. Third, we develop a system to ensure human evaluation of generative vision models are reliable, affordable, and grounded in psychophysics theory. We conclude with future opportunities for Human–Computer Interaction to aid Computer Vision.
E-commerce relies on the network, business activities can be online, so more convenient, but the online environment also caused some limits to business activities and influence, is one of the main business personnel with consumers face to face, so the business personnel can’t accurate understanding of consumer demand, business activities, the phenomenon of the accuracy is not high. At the same time, many modern enterprises to increase sales, e-commerce will pass the information to recommend ways to push their own goods to every customer, this approach does play a certain effect, but the form belongs to “wide net fishing,” failed to solve the problem of accuracy is not high at the core of the business, and because the application for a long time, so many consumers have certain resistance, Therefore, how to improve the accuracy of product recommendation is a problem worth thinking about. There is a solution to this problem under the current technology level. In order to solve this problem, it is necessary to carry out relevant research.
Purpose
As the number of joined service providers (SPs) in knowledge-intensive crowdsourcing (KI-C) continues to rise, there is an information overload problem for KI-C platforms and consumers to identify qualified SPs to complete tasks. To this end, this paper aims to propose a quality of service (QoS) evaluation framework for SPs in KI-C to effectively and comprehensively characterize the QoS of SPs, which can aid the efficient selection of qualified SPs.
Design/methodology/approach
By literature summary and discussion with the expert team, a QoS evaluation indicator system for SPs in KI-C based on the SERVQUAL model is constructed. In addition, the Decision Making Trial and Evaluation Laboratory (DEMATEL) method is used to obtain evaluation indicators' weights. The SPs are evaluated and graded by the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) and rank–sum ratio (RSR), respectively.
Findings
A QoS evaluation indicator system for SPs in KI-C incorporating 13 indicators based on SERVQUAL has been constructed, and a hybrid methodology combining DEMATEL, TOPSIS and RSR is applied to quantify and visualize the QoS of SPs.
Originality/value
The QoS evaluation framework for SPs in KI-C proposed in this paper can quantify and visualize the QoS of SPs, which can help the crowdsourcing platform to realize differentiated management for SPs and assist SPs to improve their shortcomings in a targeted manner. And this is the first paper to evaluate SPs in KI-C from the prospect of QoS.
In order to support advanced collaborations among smart products, services, users and service providers in a smart product and service ecosystem (S-PSS), this paper proposed a service-oriented hybrid digital twin (DT) and digital thread platform-based approach with embedded crowd-/service-sourcing mechanism for enabling advanced manufacturing services. This approach is well supported by the ecosystem interaction intelligence of digitally connected products, services, users, and service providers via Internet of Beings (IoB) (Things, Users and Service providers). First, driven by industrial application needs in heating industry, a conceptual model of the service-oriented hybrid platform integrated with crowdsourcing mechanism is developed, which supports the concepts of product DT, service DT and human user DT. Second, the key system realization techniques are developed to integrate service crowdsourcing and service recommendation for realizing smart services. Finally, a case study is carried out for evaluating and confirming its feasibility.
Crowdsourcing allows to build hybrid online platforms that combine scalable information systems with the power of human intelligence to complete tasks that are difficult to tackle for current algorithms. Examples include hybrid database systems that use the crowd to fill missing values or to sort items according to subjective dimensions such as picture attractiveness. Current approaches to Crowdsourcing adopt a pull methodology where tasks are published on specialized Web platforms where workers can pick their preferred tasks on a first-come-first-served basis. While this approach has many advantages, such as simplicity and short completion times, it does not guarantee that the task is performed by the most suitable worker. In this paper, we propose and extensively evaluate a different Crowdsourcing approach based on a push methodology. Our proposed system carefully selects which workers should perform a given task based on worker profiles extracted from social networks. Workers and tasks are automatically matched using an underlying categorization structure that exploits entities extracted from the task descriptions on one hand, and categories liked by the user on social platforms on the other hand. We experimentally evaluate our approach on tasks of varying complexity and show that our push methodology consistently yield better results than usual pull strategies. Copyright is held by the International World Wide Web Conference Committee (IW3C2).
The term Linked Data refers to a set of best practices for publishing and connecting structured data on the Web. These best practices have been adopted by an increasing number of data providers over the last three years, leading to the creation of a global data space containing billions of assertions-the Web of Data. In this article we present the concept and technical principles of Linked Data, and situate these within the broader context of related technological developments. We describe progress to date in publishing Linked Data on the Web, review applications that have been developed to exploit the Web of Data, and map out a research agenda for the Linked Data community as it moves forward.
Crowdsourcing allows to build hybrid online platforms that combine scalable information systems with the power of human intelligence to complete tasks that are difficult to tackle for current algorithms. Examples include hybrid database systems that use the crowd to fill missing values or to sort items according to subjective dimensions such as picture attractiveness. Current approaches to Crowdsourcing adopt a pull methodology where tasks are published on specialized Web platforms where workers can pick their preferred tasks on a first-come-first-served basis. While this approach has many advantages, such as simplicity and short completion times, it does not guarantee that the task is performed by the most suitable worker. In this paper, we propose and extensively evaluate a different Crowdsourcing approach based on a push methodology. Our proposed system carefully selects which workers should perform a given task based on worker profiles extracted from social networks. Workers and tasks are automatically matched using an underlying categorization structure that exploits entities extracted from the task descriptions on one hand, and categories liked by the user on social platforms on the other hand. We experimentally evaluate our approach on tasks of varying complexity and show that our push methodology consistently yield better results than usual pull strategies.
In crowdsourcing systems, task recommendation can help workers to find their right tasks faster as well as help requesters to receive good quality output quicker. However, previously proposed classification approach does not consider the dynamic scenarios of new workers and new tasks in the system. In this paper, we propose a Task Recommendation (TaskRec) framework based on a unified probabilistic matrix factorization, aiming to recommend tasks to workers in dynamic scenarios. Unlike traditional recommendation systems, workers do not provide their ratings on tasks in crowdsourcing systems, and thus we propose to transform worker behaviors into ratings. Complexity analysis shows that our framework is efficient and is scalable to large datasets. Finally, we conduct experiments on real-world datasets for performance evaluation.
Fuelled by increased interest among organisations to team up with customers and partners, software for supporting open innovation is emerging. In this paper, we explore how current open innovation software are designed, by whom they are used and reflect on their potential to support open innovation processes. We classified a sample of 51 systems and found, based on a correlation analysis, that most open innovation software focus on supporting online communities of innovation and some software support online marketplaces of innovation. A vast majority of open innovation software focuses on the front end of open innovation through an emphasis on the collection of ideas or problem solutions requested by authorities in technology industries. In the end of the paper, we suggest key questions that need to be addressed in order to design the next wave of open innovation software.
Crowdsourcing has become a popular approach for capitalizing on the potential of large and open crowds of people external to the organization. While crowdsourcing as a phenomenon is studied in a variety of fields, research mostly focuses on isolated aspects and little is known about the integrated design of crowdsourcing efforts. We introduce a socio-technical systems perspective on crowdsourcing, which provides a deeper understanding of the components and relationships in crowdsourcing systems. By considering the function of crowdsourcing systems within their organizational context, we develop a typology of four distinct system archetypes. We analyze the characteristics of each type and derive a number of design requirements for the respective system components. The paper lays a foundation for IS-based crowdsourcing research, channels related academic work, and helps guiding the study and design of crowdsourcing information systems.
Crowd sourcing is evolving as a distributed problem-solving and business production model in recent years. In crowd sourcing paradigm, tasks are distributed to networked people to complete such that a company¡¦s production cost can be greatly reduced. A crowd sourcing process involves operations of both requesters and workers. A requester submits a task request, a worker selects and completes a task, and the requester only pays the worker for the successful completion of the task. Obviously, it is not efficient that the amount of time spent on selecting a task is comparable with that spent on working on a task, but the monetary reward of a task is just a small amount. Literature mainly focused on exploring what type of tasks can be deployed to the crowd and analyzing the performance of crowd sourcing platforms. However, no existing work investigates on how to support workers to select tasks on crowd sourcing platforms easily and effectively. In this paper, we propose a novel idea on task matching in crowd sourcing to motivate workers to keep on working on crowd sourcing platforms in long run. The idea utilizes the past task preference and performance of a worker to produce a list of available tasks in the order of best matching with the worker during his task selection stage. It aims to increase the efficiency of task completion. We present some preliminary experimental results in case studies. Finally, we address the possible challenges and discuss the future directions.
Various forms of open sourcing to the online population are establishing themselves as cheap, effective methods of getting work done. These have revolutionised the traditional methods for innovation and have contributed to the enrichment of the concept of 'open innovation'. To date, the literature concerning this emerging topic has been spread across a diverse number of media, disciplines and academic journals. This paper attempts for the first time to survey the emerging phenomenon of open outsourcing of work to the internet using 'cloud computing'. The paper describes the volunteer origins and recent commercialisation of this business service. It then surveys the current platforms, applications and academic literature. Based on this, a generic classification for crowdsourcing tasks and a number of performance metrics are proposed. After discussing strengths and limitations, the paper concludes with an agenda for academic research in this new area.
Community Question Answering (CQA) websites provide a rapidly growing source of information in many areas. This rapid growth, while offering new opportunities, puts forward new challenges. In most CQA implementations there is little effort in directing new questions to the right group of experts. This means that experts are not provided with questions matching their expertise, and therefore new matching questions may be missed and not receive a proper answer. We focus on finding experts for a newly posted question. We investigate the suitability of two statistical topic models for solving this issue and compare these methods against more traditional Information Retrieval approaches. We show that for a dataset constructed from the Stackoverflow website, these topic models outperform other methods in retrieving a candidate set of best experts for a question. We also show that the Segmented Topic Model gives consistently better performance compared to the Latent Dirichlet Allocation Model.
Community-based Question and Answering (CQA) services have brought users to a new era of knowledge dissemination by allowing users to ask questions and to answer other users' questions. However, due to the fast increasing of posted questions and the lack of an effective way to find interesting questions, there is a serious gap between posted questions and potential answerers. This gap may degrade a CQA service's performance as well as reduce users' loyalty to the system. To bridge the gap, we present a new approach to Question Routing, which aims at routing questions to participants who are likely to provide answers. We consider the problem of question routing as a classification task, and develop a variety of local and global features which capture different aspects of questions, users, and their relations. Our experimental results obtained from an evaluation over the Yahoo!~Answers dataset demonstrate high feasibility of question routing. We also perform a systematical comparison on how different types of features contribute to the final results and show that question-user relationship features play a key role in improving the overall performance.
Online labor marketplaces offer the potential to automate tasks too difficult for computers, but don't always provide accurate results. MobileWorks is a crowd platform that departs from the marketplace model to provide robust, high-accuracy results using three new techniques. A dynamic work-routing system identifies expertise in the crowd and ensures that posted work completes within a bounded time with fair wages. A peer-management system helps prevent wrong answers. Last, social interaction techniques let best workers manage and teach other crowd members. This process allows the crowd to collaboratively learn how to solve new tasks.
We present a direction of research on principles and methods that can enable general problem solving via human compu-tation systems. A key challenge in human computation is the effective and efficient coordination of problem solving. While simple tasks may be easy to partition across individ-uals, more complex tasks highlight challenges and oppor-tunities for more sophisticated coordination and optimiza-tion, leveraging such core notions as problem decomposi-tion, subproblem routing and solution, and the recomposi-tion of solved subproblems into solutions. We discuss the in-terplay between algorithmic paradigms and human abilities, and illustrate through examples how members of a crowd can play diverse roles in an organized problem-solving pro-cess, serving not only as 'data oracles' at the endpoints of computation, but also as modules for decomposing prob-lems, controlling the algorithmic progression, and perform-ing human program synthesis.
This paper introduces a framework for conducting and writing an effective literature review. The target audience for the framework includes information systems (IS) doctoral students, novice IS researchers, and other IS researchers who are constantly struggling with the development of an effective literature-based foundation for a proposed research. The proposed framework follows the systematic data processing approach comprised of three major stages: 1) inputs (literature gathering and screening), 2) processing (following Bloom's Taxonomy), and 3) outputs (writing the literature review). This paper provides the rationale for developing a solid literature review including detailed instructions on how to conduct each stage of the process proposed. The paper concludes by providing arguments for the value of an effective literature review to IS research.
In order to understand how a labor market for human com-putation functions, it is important to know how workers search for tasks. This paper uses two complementary meth-ods to gain insight into how workers search for tasks on Mechanical Turk. First, we perform a high frequency scrape of 36 pages of search results and analyze it by looking at the rate of disappearance of tasks across key ways Mechanical Turk allows workers to sort tasks. Second, we present the results of a survey in which we paid workers for self-reported information about how they search for tasks. Our main find-ings are that on a large scale, workers sort by which tasks are most recently posted and which have the largest number of tasks available. Furthermore, we find that workers look mostly at the first page of the most recently posted tasks and the first two pages of the tasks with the most available instances but in both categories the position on the result page is unimportant to workers. We observe that at least some employers try to manipulate the position of their task in the search results to exploit the tendency to search for recently posted tasks. On an individual level, we observed workers searching by almost all the possible categories and looking more than 10 pages deep. For a task we posted to Mechanical Turk, we confirmed that a favorable position in the search results do matter: our task with favorable posi-tioning was completed 30 times faster and for less money than when its position was unfavorable.
The word Crowdsourcing –a compound contraction of Crowd and Outsourcing, was used by Howe in order to define outsourcing to the crowd. Beyond cost, benefits for the company can be substantial. It can externalize the risk of failure and it only pays for products or services that meet its expectations. The Crowdsourcing phenomenon covers heterogeneous situations and it has inspired a number of authors. However, we are still lacking a general and synthetic view of this concept. The aim of our work is to characterize Crowdsourcing in its various aspects. First we define of Crowdsourcing, and provide examples that illustrate the diversity of Crowdsourcing practices and we present similarities and differences between Crowdsourcing and established theories (Open Innovation, User Innovation and Open Source Software). Then, we propose and illustrate a typology of Crowdsourcing practices based on two criteria: the integrative or selective nature of the process and the type of tasks that are crowd sourced (simple, complex and creative tasks). In either case, the client firm seeks to mobilize external competencies. Relying upon the crowd can be an adequate method, because of its unique characteristics that are fostered by the Internet. Finally, we present some potential benefits and pitfalls of Crowdsourcing.
JEL Codes: O32, L17, L24, L29, M19
Recommender Systems (RSs) are software tools and techniques providing suggestions for items to be of use to a user. In this
introductory chapter we briefly discuss basic RS ideas and concepts. Our main goal is to delineate, in a coherent and structured
way, the chapters included in this handbook and to help the reader navigate the extremely rich and detailed content that the
handbook offers.
Collaborative recommender systems are vulnerable to malicious users who seek to bias their output, causing them to recommend
(or not recommend) particular items. This problem has been an active research topic since 2002. Researchers have found that
the most widely-studied memory-based algorithms have significant vulnerabilities to attacks that can be fairly easily mounted.
This chapter discusses these findings and the responses that have been investigated, especially detection of attack profiles
and the implementation of robust recommendation algorithms.
http://josquin.cti.depaul.edu/~rburke/pubs/burke-etal-handbook10.pdf
The new generation of Web applications known as (STS) is successfully established and poised for continued growth. STS are
open and inherently social; features that have been proven to encourage participation. But while STS bring new opportunities,
they revive old problems, such as information overload. Recommender Systems are well known applications for increasing the
level of relevant content over the “noise” that continuously grows as more and more content becomes available online. In STS
however, we face new challenges. Users are interested in finding not only content, but also tags and even other users. Moreover,
while traditional recommender systems usually operate over 2-way data arrays, STS data is represented as a third-order tensor
or a hypergraph with hyperedges denoting (user, resource, tag) triples. In this chapter, we survey the most recent and state-of-the-art
work about a whole new generation of recommender systems built to serve STS.We describe (a) novel facets of recommenders for
STS, such as user, resource, and tag recommenders, (b) new approaches and algorithms for dealing with the ternary nature of
STS data, and (c) recommender systems deployed in real world STS. Moreover, a concise comparison between existing works is
presented, through which we identify and point out new research directions.
Recommender systems form an extremely diverse body of technologies and approaches. The chapter aims to assist researchers
and developers to identify the recommendation technologies that are most likely to be applicable to different domains of recommendation.
Unlike other taxonomies of recommender systems, our approach is centered on the question of knowledge: what knowledge does
a recommender system need in order to function, and where does that knowledge come from? Different recommendation domains
(books vs condominiums, for example) provide different opportunities for the gathering and application of knowledge. These
considerations give rise to a mapping between domain characteristics and recommendation technologies.
Recommender systems are now popular both commercially and in the research community, where many approaches have been suggested for providing
recommendations. In many cases a system designer that wishes to employ a recommendation system must choose between a set of
candidate approaches. A first step towards selecting an appropriate algorithm is to decide which properties of the application
to focus upon when making this choice. Indeed, recommendation systems have a variety of properties that may affect user experience,
such as accuracy, robustness, scalability, and so forth. In this paper we discuss how to compare recommenders based on a set
of properties that are relevant for the application. We focus on comparative studies, where a few algorithms are compared
using some evaluation metric, rather than absolute benchmarking of algorithms. We describe experimental settings appropriate
for making choices between algorithms. We review three types of experiments, starting with an offline setting, where recommendation
approaches are compared without user interaction, then reviewing user studies, where a small group of subjects experiment
with the system and report on the experience, and finally describe large scale online experiments, where real user populations
interact with the system. In each of these cases we describe types of questions that can be answered, and suggest protocols
for experimentation. We also discuss how to draw trustworthy conclusions from the conducted experiments. We then review a
large set of properties, and explain how to evaluate systems given relevant properties. We also survey a large set of evaluation
metrics in the context of the properties that they evaluate.
Recommender systems have the effect of guiding users in a personalized way to interesting objects in a large space of possible
options. Content-based recommendation systems try to recommend items similar to those a given user has liked in the past. Indeed, the basic process
performed by a content-based recommender consists in matching up the attributes of a user profile in which preferences and
interests are stored, with the attributes of a content object (item), in order to recommend to the user new interesting items.
This chapter provides an overview of content-based recommender systems, with the aim of imposing a degree of order on the
diversity of the different aspects involved in their design and implementation. The first part of the chapter presents the
basic concepts and terminology of contentbased recommender systems, a high level architecture, and their main advantages and
drawbacks. The second part of the chapter provides a review of the state of the art of systems adopted in several application
domains, by thoroughly describing both classical and advanced techniques for representing items and user profiles. The most
widely adopted techniques for learning user profiles are also presented. The last part of the chapter discusses trends and
future research which might lead towards the next generation of systems, by describing the role of User Generated Content
as a way for taking into account evolving vocabularies, and the challenge of feeding users with serendipitous recommendations,
that is to say surprisingly interesting items that they might not have otherwise discovered.
Among collaborative recommendation approaches, methods based on nearest-neighbors still enjoy a huge amount of popularity,
due to their simplicity, their efficiency, and their ability to produce accurate and personalized recommendations. This chapter
presents a comprehensive survey of neighborhood-based methods for the item recommendation problem. In particular, the main
benefits of such methods, as well as their principal characteristics, are described. Furthermore, this document addresses
the essential decisions that are required while implementing a neighborhood-based recommender system, and gives practical
information on how to make such decisions. Finally, the problems of sparsity and limited coverage, often observed in large
commercial recommender systems, are discussed, and a few solutions to overcome these problems are presented.
The term "Linked Data" refers to a set of best practices for publishing and connecting structured data on the Web. These best practices have been adopted by an increasing number of data providers over the last three years, leading to the creation of a global data space containing billions of assertions-the Web of Data. In this article, the authors present the concept and technical principles of Linked Data, and situate these within the broader context of related technological developments. They describe progress to date in publishing Linked Data on the Web, review applications that have been developed to exploit the Web of Data, and map out a research agenda for the Linked Data community as it moves forward.
Yahoo! Answers is currently one of the most popular question answering systems. We claim however that its user experience could be significantly improved if it could route the "right question" to the "right user." Indeed, while some users would rush answering a question such as "what should I wear at the prom?," others would be upset simply being exposed to it. We argue here that Community Question Answering sites in general and Yahoo! Answers in particular, need a mechanism that would expose users to questions they can relate to and possibly answer.
We propose here to address this need via a multi-channel recommender system technology for associating questions with potential answerers on Yahoo! Answers. One novel aspect of our approach is exploiting a wide variety of content and social signals users regularly provide to the system and organizing them into channels. Content signals relate mostly to the text and categories of questions and associated answers, while social signals capture the various user interactions with questions, such as asking, answering, voting, etc. We fuse and generalize known recommendation approaches within a single symmetric framework, which incorporates and properly balances multiple types of signals according to channels. Tested on a large scale dataset, our model exhibits good performance, clearly outperforming standard baselines.
This paper investigates a ground-breaking incorporation of question category to Question Routing (QR) in Community Question Answering (CQA) services. The incorporation of question category was designed to estimate answerer expertise for routing questions to potential answerers. Two category-sensitive Language Models (LMs) were developed with large-scale real world data sets being experimented. Results demonstrated that higher accuracies of routing questions with lower computational costs were achieved, relative to traditional Query Likelihood LM (QLLM), state-of-the-art Cluster-Based LM (CBLM) and the mixture of Latent Dirichlet Allocation and QLLM (LDALM).
In human computation systems, humans and computers work together to solve hard problems. Many of these systems use crowdsourcing marketplaces to recruit workers. However, the task selection process of crowdsourcing workers is still unclear. We therefore outline this process and propose a structural model showing the criteria that workers use to choose tasks. The model is based on the person-job fit theory, which includes the measures demands-abilities fit and needs-supplies fit, in order to explain the work intention. We adapt the needs-supplies fit to the specific requirements of crowdsourcing markets by adding concepts for payment fit, enjoyment fit, and time fit. We further assume that the task presentation can have an effect on work intention. In this research-in-progress paper, we present our measures and experimental design as well as our newly developed method for participants' recruitment. Our work could have strong implications for organizations using crowdsourcing marketplaces.
Intrinsic and extrinsic types of motivation have been widely studied, and the distinction between them has shed important light on both developmental and educational practices. In this review we revisit the classic definitions of intrinsic and extrinsic motivation in light of contemporary research and theory. Intrinsic motivation remains an important construct, reflecting the natural human propensity to learn and assimilate. However, extrinsic motivation is argued to vary considerably in its relative autonomy and thus can either reflect external control or true self-regulation. The relations of both classes of motives to basic human needs for autonomy, competence and relatedness are discussed.
We describe an approach towards automatic, dynamic and time-critical support for competency management and expertise search through topic extraction from scientific publications. In the use case we present, we focus on the automatic extraction of scientific topics and technologies from publicly available publications using web sites like Google Scholar. We discuss an experiment for our own organization, DFKI, as example of a knowledge organization. The paper presents evaluation results over a sample of 48 DFKI researchers that responded to our request for a-posteriori evaluation of automatically extracted topics. The results of this evaluation are encouraging and provided us with useful feedback for further improving our methods. The extracted topics can be organized in an association network that can be used further to analyze how competencies are interconnected, thereby enabling also a better exchange of expertise and competence between researchers.
Crowdsourcing is an umbrella term for a variety of approaches that tap into the potential of a large and open crowd of people. So far, there is no systematic understanding of the processes used to source and aggregate contributions from the crowd. In particular, crowdsourcing organizations striving to achieve a specific goal should be able to evaluate the mechanisms that impact these processes. Following a method of IS taxonomy development we propose a new taxonomic framework for crowdsourcing processes. In contrast to previous work, this classification scheme focuses exclusively on an organizational perspective and on the mechanisms available to these organizations. The resulting dimensions are preselection of contributors, accessibility of peer contributions, aggregation of contributions, and remuneration for contributions. By classifying the processes of 46 crowdsourcing examples, we identify 19 distinct process types. A subsequent cluster analysis shows general patterns among these types and indicates a link to certain applications of crowdsourcing.
An approach to automatically recommending question based on user-word model is proposed. We first employ language modeling approach to map the relationship between a user and a question into the relationship between the user and words in the question. A use-word model is then designed to reveal and quantify the affinity relationship between users and words in the corpus. In the recommendation model, a new question is assigned to users based on the evaluation of question-user relationship. The user who has the strongest relationship with the question is recommended to answer the question. We also implement an incremental update model which can dynamically maintain the user-word model. 216,563 questions (spreading into 30 categories) from Yahoo! Answers are collected as dataset and preliminary experiments show our approach achieves question recommendation accuracy by 85.2%, which exceeds baseline methods.
We explore the problem of assigning heterogeneous tasks to workers with different, unknown skill sets in crowdsourcing markets such as Amazon Mechanical Turk. We first formal-ize the online task assignment problem, in which a requester has a fixed set of tasks and a budget that specifies how many times he would like each task completed. Workers arrive one at a time (with the same worker potentially arriving multiple times), and must be assigned to a task upon arrival. The goal is to allocate workers to tasks in a way that maximizes the to-tal benefit that the requester obtains from the completed work. Inspired by recent research on the online adwords problem, we present a two-phase exploration-exploitation assignment algorithm and prove that it is competitive with respect to the optimal offline algorithm which has access to the unknown skill levels of each worker. We empirically evaluate this al-gorithm using data collected on Mechanical Turk and show that it performs better than random assignment or greedy al-gorithms. To our knowledge, this is the first work to extend the online primal-dual technique used in the online adwords problem to a scenario with unknown parameters, and the first to offer an empirical validation of an online primal-dual algo-rithm.
Firms can seek innovative external ideas and solutions to business tasks by sponsoring co-creation activities such as crowdsourcing. To get optimal solutions from crowdsourcing contest participants, firms need to improve task design and motivate contest solvers' participation in the co-creation process. Based on the theory of extrinsic and intrinsic motivation as well as the theory of job design, we developed a research model to explain participation in crowdsourcing contests, as well as the effects of task attributes on intrinsic motivation. Subjective and objective data were collected from 283 contest solvers at two different time points. We found that intrinsic motivation was more important than extrinsic motivation in inducing participation. Contest autonomy, variety, and analyzability were positively associated with intrinsic motivation, whereas contest tacitness was negatively associated with intrinsic motivation. The findings suggest a balanced view of extrinsic and intrinsic motivation in order to encourage participation in crowdsourcing. We also suggest that crowdsourcing contest tasks should preferably be highly autonomous, explicitly specified, and less complex, as well as require a variety of skills.
Community Question Answering (CQA) service which enables users to ask and answer questions have emerged popular on the web. However, lots of questions usually can't be resolved by appropriate answerers effectively. To address this problem, we present a novel approach to recommend users who are most likely to be able to answer the new question. Differently with previous methods, this approach utilizes the inherent semantic relations among asker-question-answerer simultaneously and perform the Answerer Recommendation task based on tensor factorization. Experimental results on two real-world CQA dataset show that the proposed method is able to recommend appropriate answerers for new questions and outperforms other state-of-the-art approaches.
Crowdsourcing in the form of human-based electronic services (people services) provides a powerful way of outsourcing tasks to a large crowd of remote workers over the Internet. Research has shown that multiple redundant results delivered by different workers can be aggregated in order to achieve a reliable result. However, basic implementations of this approach are rather inefficient as they multiply the effort for task execution and are not able to guarantee a certain quality level. In this paper, we are addressing these chal- lenges by elaborating on a statistical approach for quality management of people services which we had previously proposed. The approach combines elements of statistical qual- ity management with dynamic group decisions. We present a comprehensive statistical model that enhances our original work and makes it more transparent. We also provide an extendible toolkit that implements our model and facilitates its application to real- time experiments as well as to simulations. A quantitative analysis based on an optical character recognition (OCR) scenario confirms the efficiency and reach of our model.
A majority of extant literature on recommender systems assume the input data as a given to generate recommendations. Both implicit and/or explicit data are used as input in these systems. The existence of various challenges in using such input data including those associated with strategic source manipulations, sparse matrix, state data, among others, are sometimes acknowledged. While such input data are also known to be rife with various forms of bias, to our knowledge no explicit attempt is made to correct or compensate for them in recommender systems. We consider a specific type of bias that is introduced in online product reviews due to the sequence in which these reviews are written. We model several scenarios in this context and study their properties.
With increasing knowledge demands and limited availability of expertise and resources within organizations, professionals often rely on external sources when seeking knowledge. Online knowledge communities are Internet based virtual communities that specialize in knowledge seeking and sharing. They provide a virtual media environment where individuals with common interests seek and share knowledge across time and space. A large online community may have millions of participants who have accrued a large knowledge repository with millions of text documents. However, due to the low information quality of user-generated content, it is very challenging to develop an effective knowledge management system for facilitating knowledge seeking and sharing in online communities. Knowledge management literature suggests that effective knowledge management should make accessible not only written knowledge but also experts who are a source of information and can perform a given organizational or social function. Existing expert finding systems evaluate one's expertise based on either the contents of authored documents or one's social status within his or her knowledge community. However, very few studies consider both indicators collectively. In addition, very few studies focus on virtual communities where information quality is often poorer than that in organizational knowledge repositories. In this study we propose a novel expert finding algorithm, ExpertRank, that evaluates expertise based on both document-based relevance and one's authority in his or her knowledge community. We modify the PageRank algorithm to evaluate one's authority so that it reduces the effect of certain biasing communication behavior in online communities. We explore three different expert ranking strategies that combine document-based relevance and authority: linear combination, cascade ranking, and multiplication scaling. We evaluate ExpertRank using a popular online knowledge community. Experiments show that the proposed algorithm achieves the best performance when both document-based relevance and authority are considered.
Recommender systems are widely deployed to provide user purchasing suggestion on eCommerce websites. The technology that has been adopted by most recommender systems is collaborative filtering. However, with the open nature of collaborative filtering recommender systems, they suffer significant vulnerabilities from being attacked by malicious raters, who inject profiles consisting of biased ratings.In recent years, several attack detection algorithms have been proposed to handle the issue. Unfortunately, their applications are restricted by various constraints. PCA-based methods while having good performance on paper, still suffer from missing values that plague most user–item matrixes. Classification-based methods require balanced numbers of attacks and normal profiles to train the classifiers. The detector based on SPC (Statistical Process Control) assumes that the rating probability distribution for each item is known in advance. In this research, Beta-Protection (βPβP) is proposed to alleviate the problem without the abovementioned constraints. βPβP grounds its theoretical foundation on Beta distribution for easy computation and has stable performance when experimenting with data derived from the public websites of MovieLens.
Two paradigms characterize much of the research in the Information Systems discipline: behavioral science and design science. The behavioral-science paradigm seeks to develop and verify theories that explain or predict human or organizational behavior. The design-science paradigm seeks to extend the boundaries of human and organizational capabilities by creating new and innovative artifacts. Both paradigms are foundational to the IS discipline, positioned as it is at the confluence of people, organizations, and technology. Our objective is to describe the performance of design-science research in Information Systems via a concise conceptual framework and clear guidelines for understanding, executing, and evaluating the research. In the design-science paradigm, knowledge and understanding of a problem domain and its solution are achieved in the building and application of the designed artifact. Three recent exemplars in the research literature are used to demonstrate the application of these guidelines. We conclude with an analysis of the challenges of performing high-quality design-science research in the context of the broader IS community.
A fundamental problem in many disciplines is the classification of objects in a domain of interest into a taxonomy. Developing a taxonomy, however, is a complex process that has not been adequately addressed in the information systems (IS) literature. The purpose of this paper is to present a method for taxonomy development that can be used in IS. First, this paper demonstrates through a comprehensive literature survey that taxonomy development in IS has largely been ad hoc. Then the paper defines the problem of taxonomy development. Next, the paper presents a method for taxonomy development that is based on taxonomy development literature in other disciplines and shows that the method has certain desirable qualities. Finally, the paper demonstrates the efficacy of the method by developing a taxonomy in a domain in IS.
Open source projects rely heavily on online forums as a key input to the requirements process. These forums are valuable sources for information about the users and their needs. Part of the success of open source projects depends on the collaboration and synergy of community members as they engage in active and productive discussions through posting comments, questions, and advice to online forums. However, the lack of feedback which occurs when initial posts go unanswered can negatively affect the users' perception of the project, and can subsequently impede adoption, create frustration, and lead to loss of opportunities from not understanding and satisfying the users' needs. This problem is quite common in open source forums. Our recent analysis of seven open source projects found that anywhere from 14% to 37% of user posts never get a reply. This paper directly addresses the problem of unanswered posts by presenting a hybrid recommender system that can be used to identify potential users who might be capable of responding to unanswered posts. The proposed system was evaluated using a statistical cross validation, and results show that it significantly outperformed a benchmark random recommender in terms of precision and recall. In addition, an informal analysis of the relationships between the users and the threads is presented to provide further evidence for the potential of recommender systems in this area.
In this paper, we propose a generative model, the Topic-based User Interest (TUI) model, to capture the user interest in the User-Interactive Question Answering (UIQA) systems. Specifically, our method aims to model the user interest in the UIQA systems with latent topic method, and extract interests for users by mining the questions they asked, the categories they participated in and relevant answer providers. We apply the TUI model to the application of question recommendation, which automatically recommends to certain user appropriate questions he might be interested in. Data collection from Yahoo! Answers is used to evaluate the performance of the proposed model in question recommendation, and the experimental results show the effectiveness of our proposed model.
Crowdsourcing is an online, distributed problem-solving and production model that has emerged in recent years. Notable examples of the model include Threadless, iStockphoto, InnoCentive, the Goldcorp Challenge, and user-generated advertising contests. This article provides an introduction to crowdsourcing, both its theoretical grounding and exemplar cases, taking care to distinguish crowdsourcing from open source production. This article also explores the possibilities for the model, its potential to exploit a crowd of innovators, and its potential for use beyond forprofit sectors. Finally, this article proposes an agenda for research into crowdsourcing.
Crowdsourcing is an online, distributed problem-solving and production model already in use by businesses such as Threadless.com, iStockphoto.com, and InnoCentive.com. This model, which harnesses the collective intelligence of a crowd of Web users through an open-call format, has the potential for government and non-profit applications. Yet, in order to explore new applications for the crowdsourcing model, there must be a better understanding of why crowds participate in crowdsourcing processes. Based on 17 interviews conducted via instant messenger with members of the crowd at Threadless, the present study adds qualitatively rich data on a new crowdsourcing case to an existing body of quantitative data on motivations for participation in crowdsourcing. Four primary motivators for participation at Threadless emerge from these interview data: the opportunity to make money, the opportunity to develop one's creative skills, the potential to take up freelance work, and the love of community at Threadless. A fifth theme is also discussed that addresses the language of ‘addiction’ used by the interviewees to describe their activity on the site. Understanding this kind of ‘addiction’ in an online community is perhaps the most important finding for future public crowdsourcing ventures. This study develops a more complete – though ongoing – composite of what motivates the crowd to participate in crowdsourcing applications generally, information crucial to adapt the crowdsourcing model to new forms of problem-solving.