Using Crowds for Evaluation Tasks: Validity by Numbers vs. Validity by Expertise

To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... The nature of the underlying problem is paramount for any comparative choice framework. It may relate to a variety of problem characteristics such as problem type (Felin and Zenger, 2014;Yi et al., 2012), uncertainty (Boudreau et al., 2011), novelty (Franke et al., 2013), stickiness (von Hippel, 1994), and complexity (Fernandes and Simon, 1999;Hienerth and Riar, 2014). Problem complexity has become particularly relevant for the study of innovation problems (Afuah and Tucci, 2012;Leiblein and Macher, 2009;Nickerson and Zenger, 2004). ...
Full-text available
In an era of rapid technological advances, individuals have access to a plethora of open and closed forms of problem solving. We introduce and test a framework that compares different forms of problem solving and determines how individuals make choices among them. Our analyses of 4,556 problems solved by 398 individuals reveal that high problem complexity and strong implicit social norms of open exchange increase the probability that individuals will seek external solutions rather than solve the problems themselves. Our analysis also shows that they prefer individual experts (contacted directly or via open call) over aggregated crowd knowledge. We also find that strong implicit social norms of open exchange weakened the positive impact of complexity on the probability of solving problems externally. Interestingly, this effect is more subtle when the use of aggregated crowds is compared with individual experts: while strong implicit social norms strengthen the positive impact of problem complexity when experts are contacted directly, they otherwise weaken the use of open call experts. We discuss and examine the nuanced effects between problem complexity and social norms of open exchange. The study contributes to a stream of the open innovation literature that uses a problem solving perspective to better understand under what circumstances certain forms of problem solving are utilized, and it then combines it with a novel empirical setting based on real-life experimental data.
ResearchGate has not been able to resolve any references for this publication.