Markus KrauseUniversity of California, Berkeley | UCB · International Computer Science Institute
Markus Krause
Dr. Ing.
About
59
Publications
30,175
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
749
Citations
Citations since 2017
Introduction
Additional affiliations
July 2017 - present
Alpha, Telefonica Innovation
Position
- Researcher
May 2016 - present
Mooqita
Position
- CEO
July 2015 - May 2017
Position
- PostDoc Position
Description
- Research on the Mooqita project. Matching students in online courses around the world with job opportunities.
Education
April 2008 - May 2014
October 2000 - November 2007
Publications
Publications (59)
With the advent of mobile games and the according growing and competitive market, game user research can provide valuable insights and a competitive edge if methods and procedures are employed that match the distinct challenges that mobile devices, games and usage scenarios induce. We present a summary of parameters that frame the research setup an...
Intrinsic to the transition towards, and necessary
for the success of digital platforms as a service (at
scale) is the notion of human computation. Going
beyond ‘the wisdom of the crowd’, human computation
is the engine that powers platforms and services that
are now ubiquitous like Duolingo and Wikipedia. In
spite of increasing research and popula...
Intrinsic to the transition towards, and necessary for the success of digital platforms as a service (at scale) is the notion of human computation. Going beyond 'the wisdom of the crowd', human computation is the engine that powers platforms and services that are now ubiquitous like Duolingo and Wikipedia. In spite of increasing research and popula...
Mooqita creates, delivers, and proctors a massively open online course (MOOC) in conjunction with a regional homeless shelter and local industry for people currently experiencing homelessness. It is estimated that 25% of the homeless are working or have worked within the past quarter. Additionally, around 30% of homeless in the metro area have prev...
We present a case study of Mooqita, a platform to support learners in online courses by enabling them to earn money, gather real job task experiences, and build a meaningful portfolio. This includes placing optional additional assignments in online courses. Learners solve these individual assignments, provide peer reviews for other learners, and gi...
Designers are increasingly leveraging online crowds; yet, on-line contributors may lack the expertise, context, and sensitivity to provide effective critique. Rubrics help feedback providers but require domain experts to write them and may not generalize across design domains. This paper introduces and tests a novel semi-automated method to support...
Workers' level of skill on online labor platforms varies greatly. To manage quality, requesters often decompose complex jobs into simple, repetitive, low-pay micro-tasks. Re-questers get the jobs done, but workers have little opportunity to learn and prepare for more demanding and better-pay jobs. This paper explores how we can apply learning scien...
Massive Open Online Courses (MOOCs) aim to educate the world, especially learners from developing countries. While MOOCs are certainly available to the masses, they are not yet fully accessible. Although all course content is just clicks away, deeply engaging with a MOOC requires a substantial time commitment, which frequently becomes a barrier to...
Semantic similarity or semantic relatedness are features of natural language that contribute to the challenge machines face when analyzing text. Although semantic relatedness is still a complex challenge only few ground truth data set exist. We argue that the available corpora used to evaluate the performance of natural language tools do not captur...
Massive Open Online Courses (MOOCs) aim to educate the world. More often than not, however, MOOCs fall short of this goal — a majority of learners are already highly educated (with a Bachelor degree or more) and come from specific parts of the (developed) world. Learners from developing countries without a higher degree are underrepresented, though...
Intrinsic to the transition towards, and necessary for the success of digital platforms as a service (at scale) is the notion of human computation. Going beyond ‘the wisdom of the crowd’, human computation is the engine that powers platforms and services that are now ubiquitous like Duolingo and Wikipedia. In spite of increasing research and popula...
Plagiarism in online learning environments has a detrimental effect on the trust of online courses and their viability. Automatic plagiarism detection systems do exist yet the specific situation in online courses restricts their use. To allow for easy automated grading, online assignments usually are less open and instead require students to fill i...
Education is one of the eight Millennium Development Goals (MDG) of the United Nations. Considerable interest has been displayed in online education at scale, a new arising concept to realize the MDG. Yet connecting online education to real jobs is still a challenge. This CHI workshop bridges this gap by bringing together groups and insights from r...
Expert feedback is valuable but hard to obtain for many designers. Online crowds can provide fast and affordable feedback, but workers may lack relevant domain knowledge and experience. Can expert rubrics address this issue and help novices provide expert-level feedback? To evaluate this, we conducted an experiment with a 2x2 factorial design. Stud...
Digital games are a viable alternative to accomplish crowdsourcing tasks that would traditionally require paid online labor. This study compares the quality of crowdsourc-ing with games and paid crowdsourcing for simple and complex annotation tasks in a controlled experiment. While no difference in quality was found for the simple task, paid con-tr...
Grading essays in large online classes or predicting the quality of a text submission in a crowdsourcing task is challenging. In this paper we propose a new natural language model to predict the quality of natural language responses.
Peer reviews are a viable method to handle grading in large online classes such as MOOCs or to ensure response quality in crowdsourcing. On the other hand poorly written reviews can significantly harm student and contributor satisfaction. We present a method based on our Natural Language Model to provide feedback for reviewers. We report the result...
Email is a standard and popular means of establishing potential business relationships between salespeople and future customers. Therefore, predicting if an e-mail will be successful in capturing a prospect's attention and elicit a response is an important problem for marketers and salespeo-ple alike. In this paper we propose a natural language mod...
Digital games are a viable alternative to accomplish crowdsourcing tasks that would traditionally require paid online labor. This study compares the quality of crowdsourcing with games and paid crowdsourcing for simple and complex annotation tasks in a controlled exper-iment. While no difference in quality was found for the simple task, paid contri...
We present a novel way to recognize users by the way they press a button. Our approach allows low-effort and fast interaction without the need for augmenting the user or controlling the environment. It eschews privacy concerns of methods such as fingerprint scanning. Button pressing behavior is sufficiently discriminative to allow distinguishing us...
Many MOOCs report high drop off rates for their students. Among the factors reportedly contributing to this picture are lack of motivation, feelings of isolation, and lack of interac-tivity in MOOCs. This paper investigates the potential of gamification with social game elements for increasing retention and learning success. Students in our experim...
Human behavior increasingly involves digital online software, where the activities and resources that support (1) learning, (2) work, and (3) collaboration overlap and are placed in far greater proximity than the physical world -- often just a browser-tab or window away. What scientific and practical gains in 21st century learning, work, and collab...
Fraud detection in free and natural text submissions is a major challenge for educators in general. It is even more challenging to detect plagiarism at scale and in online classes such as Massive Open Online Courses. In this paper, we introduce a novel method that analyses the writing style of an author (stylometry) to identify plagiarism. We will...
Understanding natural language is an inherently complex task for computer algorithms. Crowdsourcing natural language tasks such as semantic similarity is therefore a promising approach. In this paper, we investigate the performance of crowdworkers and compare them to offline contributors as well as to state of the art algorithms. We will illustrate...
Understanding natural language is an inherently complex task for computer algorithms. Crowdsourcing natural language tasks such as semantic similarity is therefore a promising approach. In this paper, we investigate the performance of crowdworkers and compare them to offline contributors as well as to state of the art algorithms. We will illustrate...
The first AAAI Conference on Human Computation and Crowdsourcing (HCOMP-2013) was held November 6-9, 2013, in Palm Springs, California. Three workshops took place on Saturday, November 9: Crowdsourcing at Scale (full day), Disco: Human and Machine Learning in Games (full day), and Scaling Speech, Language Understanding, and Dialogue through Crowdso...
![Fig 1. Success rates for the imitation experiment. Brennan et al. [3]...](profile/Markus-Krause-5/publication/262235536/figure/fig1/AS:392416910692366@1470570890624/Success-rates-for-the-imitation-experiment-Brennan-et-al-3-did-not-report-an-SD-for_Q320.jpg)
Ensuring authorship in online taken exams is a major challenge for e-learning in general and MOOC's in particular. In this paper, we introduce and evaluate a method to verify student identities using stylometry. We present a carefully composed feature set and use it with a K-Nearest Neighbor algorithm. We demonstrate that our method can effectively...
A recurrent challenge for human computation is motivation. Motivation is not only a prevailing topic for crowd based human computation it is also multifarious. Contributors support human computation projects for money, fun, and many other reasons. Probably the most appealing motivation from a requester’s perspective is an intrinsic interest in the...
With the advent of mobile games and the according growing and competitive market, game user research can provide valuable insights and a competitive edge if methods and procedures are employed that match the distinct challenges that mobile devices, games and usage scenarios induce. We present a summary of parameters that frame the research setup an...
Parkinson's disease causes a wide range of motor im-pairments that also affect speech. Even patients with mild speech motor disabilities do suffer from symptoms such as reduced dynamics, melody, tone, pace and continuity of speech. Besides invasive or drug based treatments, effective logopedic treatments do exist. However, constant training is a ke...
Parkinson's disease causes a wide range of motor impairments that also affect speech. Even patients with mild speech motor disabilities do suffer from symptoms such as reduced dynamics, melody, tone, pace and continuity of speech. Besides invasive or drug based treatments, effective logopedic treatments do exist. However, constant training is a key...
In recent years crowd-based and human computation systems have attracted increasing attention in science and industry. For applications that are driven by input from a multitude of human raters, ensuring data relia-bility and organizing an interactive workflow constitute a new challenge. In this paper we describe a novel approach to ensure data qua...
Designing digital games is primarily interaction design. This interaction manifests as a meaningful change in the game world. An aspect of a game can only change dynamically with a parametric model of this aspect available. One aspect of digital games is yet missing such a systematic description: the genre of a game is currently only determined by...
With recent advances in harnessing the knowledge and skill of large groups of (unknown) network-connected humans, researchers and practitioners have been designing systems that make contributions of users entertaining and more engaging. Game mechanics are being applied to the traditional human computation tasks, such as transcription, classificatio...
Mobile devices are already an important platform for digital games. These devices need specialized input metaphors as they have various restrictions such as their own hardware capabilities and the lack of external input devices. Especially challenging are fast paced interactions as in Jump and Run games. This paper explores three user inputs for Ju...
In recent years various games have been developed to gen-erate useful data for scientific and commercial purposes. Current human computation games are tailored around a task they aim to solve, adding game mechanics to conceal monotonous workflows. These gamification approaches, al-though providing valuable gaming experience, do not cover the wide r...
Research over the past years has shown that machine translation results can be greatly enhanced with the help of mono- or bilingual human contributors, e.g. by asking humans to proofread or correct outputs of machine translation systems. However, it remains difficult to determine the quality of individual revisions. This paper proposes a method to...
The scientific study of serious games is a recent development, spanning less than two decades. One aspect in this field is human computation with digital games. The core of the paradigm is to outsource problems that are not yet solvable by conventional computational systems to humans. Therefore, these problems are reformulated into tasks that are t...
Conducting surveys is a costly and time-consuming process. We developed a playful questionnaire that addresses the issue of motivation for voluntary participation. In addition, distributing playful questionnaires on Facebook gives access to basic user data, which may allow employing some of them as quality control questions or simply help to lessen...
Games for rehabilitation are developing rapidly in recent years. It has been shown that utilization of therapy and gaming technology affects positively on the patients' physical and mental condition. However, to this day there are only few playable games for Parkinson's disease patients. This paper presents the development process of WuppDi! – a co...
Motivation has been one of the central challenges of human computation. A promising approach is the integra-tion of human computation tasks into digital games. Differ-ent human computation games have been successfully dep-loyed, but tend to provide relatively narrow gaming expe-riences. This survey discusses various approaches of digital games for...
We present a human computation online game for enabling users
to contribute to the creation of a corpus of question-resource pairs
for harvesting web-based question answering. Our idea was
motivated by the popular ‘jeopardy’ quiz.
Each day millions of people play digital games with different motivations. These motivations rang from time beating to deep immersion into a narration or interacting with a community. To address all these different means, a range of game designs is necessary. Traditional human computation games cannot present all these aspects yet. This work will g...
Interactive surfaces are only just beginning to break into the market,
and they still do not offer the advanced functionality demonstrated with many
lab prototypes. The path from a prototype system to a finished product for use
in real-world scenarios is a long one, and many obstacles must be overcome.
The design of an interactive multitouch table...
Collaborative Tagging is a powerful method to create folksonomies that can be used to grasp/filter user preferences or enhance web search. Recent research has shown that depending on the number of users and the quality of user-provided tags powerful community-driven semantics or "ontologies" can emerge - as it was evident analyzing user data from s...
The present work and demonstration system aims at finding an efficient and cost-effective human computation method to expand the linguistic capabilities of interactive games that need it to respond appropriately to the language based input of their users. As a showcase scenario for the experiments conducted, we took interactive fiction applications...
Questions
Questions (5)
I am looking for literature that investigates the question how the textual description of a task influences task performance. This can be in any domain such as learning task or work task description.
It seems obvious that there must be a huge corpus of research on it. Yet it seems I am unaware of the appropriate search terms.
Links to common literature on this topic as well as ideas for terms to search on google scholar would be great.
Thanks in advance,
Markus
I have a question. In an experiment we test a treatment in three conditions (t1,t2,t3). We split a group of participants so that we have one group for each treatment. So far straight forward. However we want to estimate the interactions between two crossing variables of our participants (gender, major) and our treatment condition. Gender as well as major both have two levels.
So we have three factors:
(C)ondition (between groups, 3 levels)
(G)ender (2 levels)
(M)ajor (2 levels)
Can some one point me in the right direction how to define my model?
I do not feel confident in using: Y = C|G|M.
Thanks a lot in advance!
Markus
Projects
Project (1)
Over the past decades classic work environments have changed rapidly from manual intensive labor to cognitive demanding jobs. We see the societal impact of this development in the socio economic changes in cities with declining heavy industry such as Chicago (USA) or Manchester (UK). In recent years artificial intelligence has also begun to take over other tasks such as driving cabs, grading essays, or tracking medication. This illustrates that we need an educational system that allows workers to transition from less knowledge intensive labor to high skill jobs.
Online courses offer a scalable model for education where students from all over the world can gain access to new skills. While online courses cover topics relevant to job growth in sectors such as data science and software engineering, courses alone do not adequately prepare students to find gainful employment nor do they attract those students who can benefit the most from affordable higher education – 60% of students in online education already have at least a bachelor’s degree.
The Moocita project investigates a new model for the work learn relationship that ultimately enables students to earn money while taking courses. The model also supports students to find jobs that match their skill sets and personality. The project explores three aspects of the new work learn model 1) increasing authenticity of assignments 2) improving the learning experience and outcome and 3) effectively matching students to real-world teams and jobs. You can find more information on our project and the results of our research at www.moocita.com.
