Table 5 - uploaded by Benedikt Morschheuser
Content may be subject to copyright.
Source publication
Two parallel phenomena are gaining attention in human-computer interaction research: gamification and crowdsourcing. Because crowdsourcing's success depends on a mass of motivated crowdsourcees, crowdsourcing platforms have increasingly been imbued with motivational design features borrowed from games; a practice often called gamification. While th...
Contexts in source publication
Context 1
... with gamified crowdsolving and crowdrating were also present. However, very few cases described gamified crowd- creating systems (see Table 5). ...
Context 2
... identified 12 categories of gamification affordances (design elements, known from video games) in the reviewed body of literature (see Table 5). Points (in 53 cases) were clearly the most reported gamification components and usually provided the basis for other affordances. ...
Context 3
... reviewed literature indicates that gamified crowdsourcing systems that process homogeneous, easily enumerable tasks, such as in crowdrating or crowdprocessing, most commonly implement simple points-based and leaderboard-based game designs (Table 5). Generally, these homogeneous tasks are simple, repetitive, and are quick to complete. ...
Context 4
... overview indicates that points and leaderboards are the most used gamification affordances (Table 5). However, the differences are in the details. ...
Context 5
... the other hand, crowdcreating may benefit from mecha- nisms that reward cooperative and collaborative behavior. Several examples use rich gamification de- signs with a diverse set of affordances (see Table 5). Massung et al. (2013) and Preist et al. (2014) propose mixing several motivational affordances for different target groups to increase the overall out- come. ...
Context 6
... Table 7 shows that, in some cases, gamified crowdsourcing systems use a combination of gamification and financial incentives. Considering how gamification is implemented in crowdsourcing (see Table 5), it appears that monetary rewards have been used in implementations that employ simpler gamification designs, mainly in combination with points and leaderboards. Although studies suggest that extrinsic rewards (such as money) can potentially decrease intrinsic motivation (Deci, 1971;Deci et al., 1999), Massung et al. (2013) and Preist et al. (2014) found in their experiment that gamification in combination with financial rewards can in fact increase participation when compared to gamification alone. ...
Context 7
... overview indicated that gamification implementations differ in the context of crowdsolving, crowdrating, and crowdprocessing approaches (Table 5). Finally, we identified different recommenda- tions for designers of gamified crowdsourcing systems. ...
Context 8
... on the type of crowdsourcing (crowd- creating, crowdsolving, crowdprocessing, and crowdrating), we identified patterns in the use of gami- fication affordances. In the context of crowdsourcing initiatives that provide homogenous and often more monotonous tasks such as crowdprocessing and crowdrating, authors commonly report the use of simple forms of gamification such as points and leaderboards (Table 5). Conversely, crowdsourcing studies with crowdcreating and crowdsolving work that seek diverse and creative contributions employ gamification in more manifold ways with a richer set of mechanics. ...
Citations
... Gamification in MapSwipe was implemented by experience points the users obtain for completed tasks which are reflected in experience levels through the badges gained. This simple approach has been frequently used for crowd-sourcing applications with relatively simple repetitive tasks [39]. The user-level information about MapSwipe activity accessible for registered users is comparable to other approaches in the OSM ecosystem such as for the HOT tasking manager or "How did you contribute to OSM?" [40]. ...
Natural hazards threaten millions of people all over the world. To address this risk, exposure and vulnerability models with high resolution data are essential. However, in many areas of the world, exposure models are rather coarse and are aggregated over large areas. Although OpenStreetMap (OSM) offers great potential to assess risk at a detailed building-by-building level, the completeness of OSM building footprints is still heterogeneous. We present an approach to close this gap by means of crowd-sourcing based on the mobile app MapSwipe, where volunteers swipe through satellite images of a region collecting user feedback on classification tasks. For our application, MapSwipe was extended by a completeness feature that allows to classify a tile as “no building”, “complete” or “incomplete”. To assess the quality of the produced data, the completeness feature was applied to four regions. The MapSwipe-based assessment was compared with an intrinsic approach to quantify completeness and with the prediction of an existing model. Our results show that the crowd-sourced approach yields a reasonable classification performance of the completeness of OSM building footprints. Results showed that the MapSwipe-based assessment produced consistent estimates for the case study regions while the other two approaches showed a higher variability. Our study also revealed that volunteers tend to classify nearly completely mapped tiles as “complete”, especially in areas with a high OSM building density. Another factor that influenced the classification performance was the level of alignment of the OSM layer with the satellite imagery.
... Several concepts, such as immersion, fun, presence, participation, engagement, fluidity, play, and adaptability -as well as what causes play pleasant, such as social play -have been used to analyze user experience in games [4]. The major purpose of game development is to create games that are enjoyable to play, interesting, and full of surprises that challenge players and let them use their talents, but also provide visually appealing experiences, facilitate social connection, or let the user identify with the game. ...
... • Sound impacts make participants feel stressed out 4 Psychosis •The game employs binaural audio to give you the impression that you are hearing them as well. ...
Game designers and critics have adopted the idea of immersion to define a rich and satisfying gaming experience. The intensity to which players may engage themselves in a game experience is frequently claimed to be influenced by the game's user interface, though there are several definitions for this idea. This investigation was intended to look at the impact of immersion-based task-oriented gaming addiction on youth based on the gameplay experience of users. This research's goals are to ascertain: (a) exposure to immersion-based task-oriented gaming, (b) the levels of immersion-based task-oriented games addiction and mental health, and (c) The connection between immersions based task oriented games addiction and mental health, especially loneliness, anxiety and depression. Survey research methods were used in this investigation with the UCD model for the usability evaluation of selected immersive games. A survey questionnaire served as the study tool and a sample approach was utilized to pick the data. This survey included 210 individuals in total, ranging in age from 17 to 35. The most popular type of immersion-based task-oriented games Player unknown's battlegrounds (PUBG) and Call of duty (CoD) were selected for research. The levels of immersion-based task-oriented gaming addiction and mental health among the young generation were found to be significantly low. The connection between immersion-based task-oriented gaming addiction and mental health components, namely depression, anxiety, and loneliness, was found to be significant and positive. Jilin Daxue Xuebao (Gongxueban)/Journal of Jilin University (Engineering and Technology Edition)
... el mundo empresarial (team-building y reclutamiento de empleados entre otros), la salud pública (para inculcar buenas prácticas a los pacientes, por ejemplo) o la evaluación de servicios públicos (como la recogida de basura o el estado de las calles) (Torres Toukoumidis et al., 2017;Koivisto y Hamari, 2019;Navarro Mateos, Pérez López y Femia, 2021). Sin embargo, es en el ámbito de la educación donde más se ha profundizado esta línea de investigación (Morschheuser et al., 2017). ...
A pesar del éxito creciente de la gamificación en la universidad, resulta sorprendente que todavía no existan estudios sobre esta innovación docente en el campo de las políticas públicas. Este artículo pretende colmar este vacío. Después de recordar los fundamentos del concepto de gamificación, sus beneficios y sus limitaciones, este artículo pone de relieve una serie de actividades desarrolladas previamente: el uso de ladrillos Lego, un concurso de Kahoot, un rally urbano, un juego de supervivencia con huertos, un juego de escape, una competición de SimCity y una simulación basada en Diplomacy. Este artículo debe ser considerado como una hoja de ruta que debe ser adaptada al contexto de cada docente. Se aboga por un uso moderado y razonado de la gamificación. Esta corriente debe ser considerada por lo que es: una simple herramienta al servicio del aprendizaje, y no como un fin en sí misma.
... The authors used a game-based tool that made the task easy to complete by including points and a leaderboard on the platform. This method of so-called gamification is frequently used by crowdsourcing platforms and has been shown to increase the engagement of the crowd and improve the quality of the crowdsourced work [37]. Bittel et al [38] used a hybrid crowd-ML approach to create the largest publicly available data set of annotated endoscopic images. ...
... To collect nonexpert image annotations, we used the commercially available platform DiagnosUs (Centaur Labs) [44] through a collaboration agreement. Users can sign up to the app and participate in competitions, which increases engagement and improves accuracy [37]. Users are recruited via a referral system or advertisements on social media. ...
Background
Dermoscopy is commonly used for the evaluation of pigmented lesions, but agreement between experts for identification of dermoscopic structures is known to be relatively poor. Expert labeling of medical data is a bottleneck in the development of machine learning (ML) tools, and crowdsourcing has been demonstrated as a cost- and time-efficient method for the annotation of medical images.
Objective
The aim of this study is to demonstrate that crowdsourcing can be used to label basic dermoscopic structures from images of pigmented lesions with similar reliability to a group of experts.
Methods
First, we obtained labels of 248 images of melanocytic lesions with 31 dermoscopic “subfeatures” labeled by 20 dermoscopy experts. These were then collapsed into 6 dermoscopic “superfeatures” based on structural similarity, due to low interrater reliability (IRR): dots, globules, lines, network structures, regression structures, and vessels. These images were then used as the gold standard for the crowd study. The commercial platform DiagnosUs was used to obtain annotations from a nonexpert crowd for the presence or absence of the 6 superfeatures in each of the 248 images. We replicated this methodology with a group of 7 dermatologists to allow direct comparison with the nonexpert crowd. The Cohen κ value was used to measure agreement across raters.
Results
In total, we obtained 139,731 ratings of the 6 dermoscopic superfeatures from the crowd. There was relatively lower agreement for the identification of dots and globules (the median κ values were 0.526 and 0.395, respectively), whereas network structures and vessels showed the highest agreement (the median κ values were 0.581 and 0.798, respectively). This pattern was also seen among the expert raters, who had median κ values of 0.483 and 0.517 for dots and globules, respectively, and 0.758 and 0.790 for network structures and vessels. The median κ values between nonexperts and thresholded average–expert readers were 0.709 for dots, 0.719 for globules, 0.714 for lines, 0.838 for network structures, 0.818 for regression structures, and 0.728 for vessels.
Conclusions
This study confirmed that IRR for different dermoscopic features varied among a group of experts; a similar pattern was observed in a nonexpert crowd. There was good or excellent agreement for each of the 6 superfeatures between the crowd and the experts, highlighting the similar reliability of the crowd for labeling dermoscopic images. This confirms the feasibility and dependability of using crowdsourcing as a scalable solution to annotate large sets of dermoscopic images, with several potential clinical and educational applications, including the development of novel, explainable ML tools.
... Es desafiante imaginar que un/a ciudadano/a usa la app de forma continua, sin que exista un beneficio directo. Otras aplicaciones móviles han experimentado con elementos de gamification (Thiel y Fröhlich, 2016;Morschheuser et al., 2017), como la introducción de niveles de competencia de usuarios (desde novato a experto), o el ranking de usuarios por sus contribuciones por ciudad, la entrega de premios (como tasas o camisetas) y cupones para mejores precios en una tienda (online) si un usuario muestra un cierto nivel de actividad. En qué medida estos métodos pueden ser útil para retener usuarios de STRIDE.App queda en duda. ...
RESUMEN: La caminabilidad de calles o barrios puede ser evaluada de dos formas: por medio de modelos, usando bases de datos geográficas sobre el uso de suelo, tal como lo hace Walkscore.com, o a través de encuestas focalizadas en peatones. Ambas evaluaciones de caminabilidad ayudan a planificar barrios con mejor accesibilidad, diseñar intervenciones urbanistas y también podría entregar perspectivas del comportamiento peatonal a los investigadores y planificadores. Para facilitar los últimos dos propósitos, se desarrolla una aplicación móvil, llamada STRIDE.App, que permite a los peatones compartir sus experiencias de caminata. El enfoque del desarrollo de la aplicación ha sido una interfaz sencilla de utilizar, que logra compartir con bastante rapidez la percepción del transeúnte, al calificar la ubicación actual en tres posibles alternativas para caminar: 1. buena, 2. con algunos problemas o 3. mala-por medio de los colores verde, amarillo y rojo respectivamente. La información sobre los peatones, tales como edad, género y su capacidad física, es considerada clave para conocer "quien" ha tenido una experiencia positiva, regular o negativa. La app móvil fue usada en 4 casos de estudios en 2 continentes, comprobando su utilidad, pero también destacando los desafíos con respecto a la representatividad de los datos, y límites en el entendimiento de los contextos locales. Palabras clave caminabilidad, herramientas de recolección de datos, experiencias de caminata, seguridad de peatones.
//
ABSTRACT Walkability of a street or a neighbourhood can be evaluated in two ways, either through land-use data-based models, such as done by Walkscore.com, or through pedestrian focused surveys. Both types of evaluation help to plan for better accessibility in a neighbourhood, plan street interventions, but also, give researchers and planners insights on pedestrian behaviour. To facilitate the latter two purposes, we developed the mobile phone application STRIDE.App that permits pedestrians to share their walking experience. Focus of the app development has been an easy-to-use interface that allows to fairly quickly share one's perception by qualifying the current location as being either good (green), with some issues (yellow) or bad (red) for walking. Information on the pedestrian itself concerning age, gender, and its physical ability was considered key for knowing "who" has a positive or negative walking experience. The app was applied in 4 case studies on 2 continents, proving its utility, but also highlighting challenges. The latter include representativeness of data and a limited understanding of the local context when trying to make sense of the data.
... Significant research is required to develop a solid mechanism of theoretical as well as methodological base on which knowledge is gathered (Hamari, Koivisto, & Sarsa, 2014;Seaborn & Fels, 2015). Gamification though from time to time has been observed as coercive, suppressive, and a way to disconnect workers from their fundamental motivation (Bogost, 2015;Morschheuser et al., 2017;Bahr et al., 2022;Settanni & Srai, 2022). These understandings seem to stem from a very specific understanding of gamification and they should be recognized and investigated. ...
Supply chains are changing due to changes in the global environment and to make the strategies effective most of the companies are striving towards the trends in the supply chain. Among the trends gamification is one of those that are creating an impact in the workspace with the techniques that are associated with it. Gamification in the supply chain had the least application and this study gave a road map for the decision makers like supply chain managers with the help of reviewing papers from the databases that are available. Findings say that giants like Amazon and Starbucks are applying gamification techniques to bring more transparency and visibility to reduce errors and to mitigate errors. This paper is providing evidence from the different activities like order management, warehousing activities with case applications are discussed. We conducted systematic literature review with 118 papers on gamification and 22 found relevant to the supply chain and its activities.
... Concepts related to crowdsourcing include cocreation, open innovation, and user innovation (Chesbrough, 2006;Prahalad and Ramaswamy, 2000;Von Hippel and Katz, 2003). The benefits of crowdsourcing have been thoroughly established (Aitamurto et al., 2011;Buecheler et al., 2010;Lew, 2014;Morschheuser et al., 2017;von Ahn and Dabbish, 2008), and success stories can be found in various fields, from astronomy (e.g. Zooniverse; Simpson et al., 2014) to business. ...
... In sum, when applied in the right circumstances, to the right crowd, and using a method and motivation best suited for a specific task, crowdsourcing can deliver very useful outcomes. It is, however, important to note that successful completion of a crowdsourcing task also requires a careful analysis of the related goals, the problem-solving environment, the expertise required, complementary activities and capabilities, and the competitive environment (Aitamurto et al., 2011;Morschheuser et al., 2017;Pe-Than et al., 2015). 10 4 The crowdsourcing for language learning game − CrowLL ...
... In terms of the type of crowdsourced work, Morschheuser et al. (2017) propose a categorization of crowdsourcing types based on the framework presented by Geiger and Schader (2014). Based on this, we consider CrowLL as a crowdrating game, given that "crowdrating systems commonly seek to harness the so-called wisdom of crowds (Surowiecki, 2005) to perform collective assessments or predictions. ...
One way to stimulate the use of corpora in language education is by making pedagogically appropriate corpora, labeled with different types of problems (sensitive content, offensive language, structural problems). However, manually labeling corpora is extremely time-consuming and a better approach should be found. We thus propose a combination of two approaches to the creation of problem-labeled pedagogical corpora of Dutch, Estonian, Slovene and Brazilian Portuguese: the use of games with a purpose and of crowdsourcing for the task. We conducted initial experiments to establish the suitability of the crowdsourcing task, and used the lessons learned to design the Crowdsourcing for Language Learning (CrowLL) game in which players identify problematic sentences, classify them, and indicate problematic excerpts. The focus of this paper is on data preparation, given the crucial role that such a stage plays in any crowdsourcing project dealing with the creation of language learning resources. We present the methodology for data preparation, offering a detailed presentation of source corpora selection, pedagogically oriented GDEX configurations, and the creation of lemma lists, with a special focus on common and language-dependent decisions. Finally, we offer a discussion of the challenges that emerged and the solutions that have been implemented so far.
... Sustaining motivation of participants throughout different stages of CS engagement poses challenges for CS systems' long-term viability; however, the role of participant system-use practices has not yet received sufficient attention (Morschheuser et al., 2017;Nevo & Kotlarsky, 2020). To address this ongoing social challenge and the research gap in the CS literature, our study provided a much-needed exploration of CS participants' sustaining motivation facilitated by system-use practices. ...
The success of crowdsourcing (CS) systems depends on sustained participation, which is an ongoing challenge for the majority of CS providers. Unfortunately, participants are frequently demotivated by technical difficulties and the incorrect use of CS systems, which can result in CS failure. Although the literature generally assumes that sustained participation in CS is determined by a shift between intrinsic and extrinsic motivation, the role of system‐use practices in facilitating such a shift remains unknown. We explore how CS system‐use practices influence participants' sustained motivation, evolving from initiation to progression to sustention. Using the notion of technology‐in‐practice as a lens, we develop and examine a process model using an in‐depth case study of a large‐scale ongoing CS project, the Australian Newspaper Digitisation Program. The findings suggest that CS participants' motivation is shaped by an evolving combination of three basic components (i.e., contextual condition, outcome and action intensity) and mediated by two types of system‐use practice (i.e., passive and active). Passive‐use practices facilitate sustaining motivation from initiation to progression, whereas active‐use practices have a key role in sustention. Our study contributes to the emerging literature on the substantial role of system‐use practices in sustaining motivation, resulting in sustained participation. The findings also offer actionable insights into improving the viability of CS systems in retaining and motivating continuous and increased contributions from participants.
... Gamification has been applied across fields ranging from politics and education, over health and sustainability to crowdsourcing and public engagement (cf. Douglas & Brauer, 2021;Koivisto & Hamari, 2019;Morschheuser et al., 2017). Gamification and intuitive application design can improve engagement and retention of a heterogeneous group of participants (cf. ...
... Bubalo et al., 2019;Fritz et al., 2017;Martella et al., 2015), encourages more diverse and interesting contributions (cf. Bubalo et al., 2019;Morschheuser et al., 2017;Woodyer, 2012) and has been applied in a number of spatial applications (Morschheuser et al., 2017). The concept of ludic geographies (Woodyer, 2012) states "[playing] exploits the openness and circumstance of the everyday" (Woodyer, 2012, p. 320) and playful activities present "an opportunity for critical reflection on everyday experiences and values" (Woodyer, 2012, p. 322), highlighting the potential of using ludic systems and playful interactions to generate rich spatial data on everyday landscapes. ...
... Bubalo et al., 2019;Fritz et al., 2017;Martella et al., 2015), encourages more diverse and interesting contributions (cf. Bubalo et al., 2019;Morschheuser et al., 2017;Woodyer, 2012) and has been applied in a number of spatial applications (Morschheuser et al., 2017). The concept of ludic geographies (Woodyer, 2012) states "[playing] exploits the openness and circumstance of the everyday" (Woodyer, 2012, p. 320) and playful activities present "an opportunity for critical reflection on everyday experiences and values" (Woodyer, 2012, p. 322), highlighting the potential of using ludic systems and playful interactions to generate rich spatial data on everyday landscapes. ...
Measuring what citizens perceive and value about landscapes is important for landscape monitoring. Capturing temporal, spatial and cultural variation requires collection of data at scale. One potential proxy data source are textual descriptions of landscapes written by volunteers. We implemented a gamified application and crowdsourced a multilingual corpus of in-situ descriptions of everyday lived landscapes. Our implementation focused on the aesthetics of exploration, expression and fellowship in the mechanics, dynamics, aesthetics (MDA) framework. We collected 503 natural language landscape descriptions from 384 participants in English (69.7%), German (25.1%) and French (5.3%) and most contributions were made in urban areas (54.7%). The most frequent noun lemma in English was “tree” and in German “Fenster” (window). By comparing our English collection to corpora of everyday English and landscape descriptions, we identified frequent lemmas such as “tree”, “window”, “light”, “street”, “garden” and “sky” which occurred significantly more than expected. These terms hint as to important components of the everyday landscapes of our users. We suggest a number of ways in which our corpus could be used in ongoing research on landscapes, complementing existing PPGIS approaches, providing data for domain specific lexicons for landscape analysis and as an input to landscape character assessment.
... With respect to crowdsourcing, some reviews have been conducted. These reviews have focused on clarifying crowdsourcing definition [6,8], categorizing crowdsourcing [34], and further positioning it in relation to other concepts (such as open innovation [35], opensource software [36], and organizational forms [37]), broadly describing and exploring the development of the crowdsourcing literature [38][39][40][41], revealing the applications of crowdsourcing in different areas (such as the medical and health sectors [42], data management [43], and information systems [40]), defining the components of crowdsourcing systems and their relationships [44,45], and investigating a specific component or issue of crowdsourcing (such as privacy challenges [46], social mechanisms [47], gamification designs [48], task affordance [49], and factors influencing decisions to crowdsource [50]). The present review concentrates on solver behaviors in KI-C and is dedicated to synthesizing the factors that influence solver behaviors, which is different from past reviews on crowdsourcing. ...
Solver participation plays a critical role in the sustained development of knowledge-intensive crowdsourcing (KI-C) systems. Extant theory has highlighted numerous factors that influence solvers’ participation behaviors in KI-C. However, a structured investigation and integration of significant influential factors is still lacking. This study consolidated the state of academic research on factors that affect solver behaviors in KI-C. Based on a systematic review of the literature published from 2006 to 2021, this study identified five major solver behaviors in KI-C. Subsequently, eight solver motives and seventeen factors under four categories, i.e., task attributes, solver characteristics, requester behaviors, and platform designs, were identified to affect each of the solver behaviors. Moreover, the roles of solver motives and the identified factors in affecting solver behaviors were demonstrated as well. We also suggested a number of areas meriting future research in this study.
