Greater variability in judgements of the value of novel ideas

Abstract

Understanding the factors that hinder support for creative ideas is important because creative ideas fuel innovation—a goal prioritized across the arts, sciences and business. Here we document one obstacle faced by creative ideas: as ideas become more novel—that is, they depart more from existing norms and standards—disagreement grows about their potential value. Specifically, across multiple contexts, using both experimental methods (four studies, total n = 1,801) and analyses of archival data, we find that there is more variability in judgements of the value of more novel (versus less novel) ideas. We also find that people interpret greater variability in others’ judgements about an idea’s value as a signal of risk, reducing their willingness to invest in the idea. Our findings show that consensus about an idea’s worth diminishes the newer it is, highlighting one reason creative ideas may fail to gain traction in the social world.

Full text

NOVELTY AND VARIABILITY 1
Greater variability in judgments of the value of novel ideas
Wayne Johnsona & Devon Proudfootb
aManagement Department
Eccles School of Business University of Utah
Salt Lake City, UT 84112
bHuman Resource Studies Department
ILR School, Cornell University
Ithaca, NY, 14853

NOVELTY AND VARIABILITY 2
Abstract
Understanding the factors that hinder support for creative ideas is important because creative
ideas fuel innovation—a goal prioritized across the arts, sciences, and business. Here, we
document one obstacle faced by creative ideas: As ideas become more novel—that is, they
depart more from existing norms and standards—disagreement grows about their potential value.
Specifically, across multiple contexts, using both experimental methods and analyses of archival
data, we find that there is more variability in judgments of the value of more novel (vs. less
novel) ideas. We also find that people interpret greater variability in others’ judgments about an
idea’s value as a signal of risk, reducing their willingness to invest in the idea. Our findings show
that consensus about an idea’s worth diminishes the newer it is, highlighting one reason creative
ideas may fail to gain traction in the social world.
Keywords: novelty, creativity, variability, consensus, value judgments

NOVELTY AND VARIABILITY 3
Introduction
Society requires innovation to move forward. Innovation, in turn, requires new ideas.
However, for new ideas to successfully drive innovation, people must support and invest in
them1,2,3. In the current research, we examined a reason why new ideas may fail to gain
momentum in the social world: As ideas become more novel—that is, they depart more from
existing norms and standards—disagreement grows about their potential value. Specifically,
using both experimental and archival data, we tested whether there is more variability in
judgments of the value of more novel (vs. less novel) ideas. Critically, we also tested whether
people interpret greater variability in others’ judgments about an idea’s value as a signal of risk,
reducing their willingness to invest in the idea.
Previous research has identified several psychological and contextual factors explaining
why, despite society’s purported desire for creativity, creative ideas, once generated, often
become stalled1, 2, 4,5,6,7,8,9,10. While creative ideas are defined as those that are both novel and
useful11, research on the psychology of idea evaluation points to a bias against novelty as a factor
contributing to why creative ideas generate resistance. For instance, even in innovation-focused
contexts, evaluators have been shown to favor more familiar ideas over less familiar ones12,13,14.
Research by Mueller and colleagues7 suggests that evaluators may respond negatively to creative
ideas because of an aversion to the uncertainty associated with novelty.
We hypothesized that, in addition to the already established tendency for novel ideas to
often elicit negative reactions, evaluators may also have more varied judgments of the value of
ideas as they become more novel, with this variability itself contributing to why novel ideas
often fail to gain traction. We conceptualized value as a subjective assessment of an idea’s worth
within a given context3,11,15 and novelty as the extent to which an idea is different from current

NOVELTY AND VARIABILITY 4
templates and standards16. We limited the scope of our examination to situations in which idea
evaluations were independent (i.e., multiple independent judges rated how valuable a series of
ideas varying in novelty were). This approach allowed us to isolate the impact of novelty on
value judgment variability absent of potential peer pressure or contamination effects17.
Previous studies point to the possibility that judgments of novel ideas might be
particularly prone to variability between evaluators. First, there is evidence that creativity
judgments may be inconsistent across different groups of judges18,19,20,21. For instance, a study by
Runco and colleagues18 compared ratings of the creativity of artwork across professionals and
student judges, finding that creativity ratings differed between these two groups. Second, there is
evidence that groups often face challenges in reaching internal consensus about novel ideas6,22,23.
For instance, a qualitative study by Harvey and Mueller6 found that consensus pressures often
lead groups to reject novel ideas rather than move forward with them. Although these studies did
not directly examine whether judgments of value varied across individual evaluators more as a
function of idea novelty, these findings provide initial evidence that evaluators often disagree
when assessing creative ideas.
We theorized that judgments of value might converge less as ideas become more novel
because, for more novel ideas, evaluators have fewer common templates upon which to base
their judgments. For less novel ideas—ideas relatively similar to existing templates—evaluators
may more easily engage in a matching process, comparing the idea to familiar ideas that exist in
the mind as schemas24, We suggest that, as schemas tend to be similar among observers25,26,
there may be relative convergence in judgments of the value of ideas that appear similar to
common templates. However, for more novel ideas, this matching process may become more
difficult, as evaluators have fewer common templates to use to interpret the value of the focal

NOVELTY AND VARIABILITY 5
idea13,27. In such situations, evaluators are more likely to be influenced by idiosyncratic
knowledge, preferences, and situational factors that cause evaluators’ judgments to differ from
each other, producing greater judgmental variability28. This line of reasoning is consistent with
the notion that there is an inverse relationship between how novel an idea is and the extent to
which one can predict its usefulness29,30,31,32. Our theorizing is also supported by studies showing
that when assessing creative ideas, evaluators’ judgments are often prone to errors and may be
shaped by peripheral contextual factors and idiosyncratic individual differences, such as idea
pitcher personality and evaluator construal level3,33,34,35,36,37,38,39,40.
Notably, people often seek others’ feedback about ideas before deciding whether to
devote resources to developing them2,3,41. We propose that, independent of the content of the idea
itself, and, importantly, the average judged value of the idea by others, the degree of variability
in other people’s assessments of the potential value of an idea may itself be a cue that individuals
use when deciding whether to invest in an idea. Specifically, we hypothesized that greater
variability in evaluations may create an obstacle for creative ideas because people may interpret
higher variability in others’ assessments of an idea’s value—as we predict for more novel
ideas—as a sign that investing in the idea is risky, thus reducing people’s willingness to support
the idea42.
Results
Overview. Five studies tested our predictions. Studies 1-4 examined the relationship
between idea novelty and variability in value judgments across different contexts. Study 5
examined how variability in others’ assessments of an idea’s value impacted observers’ intent to
invest in the idea. All analyses were conducted using SPSS. All effect sizes reported were
calculated using SPSS. All statistical tests reported are two-tailed tests. To capture variability

NOVELTY AND VARIABILITY 6
(i.e., variability between evaluators in judgments of idea value), we examined the standard
deviation of value ratings for each idea evaluated28.We tested whether value standard deviation
(referred to as MSD) was larger on average for higher novelty ideas compared to lower novelty
ideas. Hence, in our studies examining this prediction (Studies 1-4), ideas were the unit of
analysis. Across all studies, we did not find a positive relationship between mean value ratings
and value standard deviation, indicating that our predicted effect is not reducible to scalar
variability—the tendency for ratings on a response scale to become more varied as ratings
increase in magnitude43. See Table S1 in the Online Supplement.
Study 1: Business Venture Pitches. In Study 1, we examined whether there was more
variability in value judgments for more novel (vs. less novel) business venture pitches. We used
descriptions of ventures pitched on the U.S. television show Shark Tank (N=1088). First, we
conducted a pretest in which we asked a convenience sample of 1,927 U.S. residents recruited
from the online platforms Prolific Academic and Amazon’s Mechanical Turk (Mturk) to rate the
novelty of each pitch. For more details on the pretest, see Methods. Next, we asked a separate
convenience sample of 1000 U.S residents recruited from Mturk to rate the value offered by the
250 pitches rated most novel and the 250 pitches rated least novel in the pretest. For each pitch,
participants were asked, “What is the degree of value offered by this idea?” As predicted, there
was more variability in value ratings for the higher novelty pitches (MSD = 1.62, SD = 0.27)
compared to the lower novelty pitches (MSD = 1.51, SD = 0.25), t(498) = 4.46, p < .001, d = .40,
95% CI for difference [.06, .15]. Also consistent with predictions, there was a positive
association between pitch novelty score (generated in our pretest) and value judgment variability,
r (498) = .198, p < .001, 95% CI [.11, .28].

NOVELTY AND VARIABILITY 7
Study 2: Films. In Study 2, we examined archival audience evaluations of films (N=523)
premiering at Sundance Film Festival from 2015-2022. Sundance offered a useful context in
which to test the predicted relationship between novelty and judgmental variability in a natural
environment because films premiering at Sundance are organized into multiple categories, with
certain films selected by festival programmers to premiere in categories specifically highlighting
more novel projects. We used this feature of the festival’s structure to examine whether there
was greater variability in audience evaluations of films premiering in more novel categories at
Sundance (i.e., the festival categories NEXT and Midnight) compared to films premiering in less
novel categories at Sundance (i.e., the festival categories U.S. Documentary, World
Documentary, U.S. Narrative, or World Narrative). For complete details on how films are sorted
into these categories by Sundance programmers and how we determined the relative novelty of
each category, see Methods. Archival audience ratings for each film, which we used as a proxy
for value, were captured using a single-item measure and were collected from: https://cannes-
ratings.tk/Sundance, which presents auto-aggregated web-based evaluations from audience
members at Sundance, typically a mix of industry insiders, film makers, journalists, and film
fans44. Ratings were aggregated in real-time during the festival.
We analyzed the data using ordinary least squares (OLS) regression with Huber-White
robust standard errors. As detailed in Figure 1 and Table 1 Model 1, consistent with predictions,
there was more variability in archival audience ratings of films within NEXT, a category at
Sundance featuring higher novelty films, compared to films in each of the lower novelty
categories (U.S. Documentary, World Documentary, U.S. Narrative, and World Narrative).
There was also more variability in archival audience ratings of films within Midnight, a category
at Sundance featuring higher novelty films, compared to films in each of the lower novelty

NOVELTY AND VARIABILITY 8
categories (U.S. Documentary, World Documentary, U.S. Narrative, and World Narrative),
although the comparison between Midnight and World Narrative did not reach statistical
significance (p = .050). There were similar levels of variability in ratings of films within NEXT
and Midnight, the two higher novelty categories. Results were robust after controlling for
number of ratings and when analyzing only films with at least 20 ratings (Table 1 Models 2 and
3), although the comparison between Midnight and World Narrative was non-significant (p =
.169) when analyzing only films with at least 20 ratings.
Figure 1: Average Standard Deviation in Film Ratings a Function of Film Category (Study 2)
Study 3: Experimental Evidence. Studies 1-2 showed our predicted effect in
evaluations of real-world ideas but did not control for covariation between idea novelty and other
factors that could impact variability in value judgments. Study 3 addressed this limitation by
0
1
2
3
U.S.
Documentary
World
Documentary
U.S.
Narrative
World
Narrative
Midnight NEXT
Average Standard Deviation in Ratings
Lower novelty film categories Higher novelty film categories

NOVELTY AND VARIABILITY 9
experimentally manipulating idea novelty, providing a causal test of the influence of novelty on
value judgment variability. We explored evaluations of abstract art, predicting more variability in
judgments of the value of a set of paintings when they were construed as more novel within a
given context than when identical paintings were construed as less novel within a given context.
We recruited 200 U.S. residents from the online platform Lucid Theorem and asked them
to imagine a particular context—an alien society on a faraway planet. Participants read that they
would assess the subjective value of a series of paintings specifically within the context of this
alien society. Participants read that in this context, a particular style of abstract art was highly
acclaimed by critics. To manipulate novelty within this context, participants were first shown a
prototype of the style of painting that was highly acclaimed and were then asked to judge the
value of a subset of 80 additional paintings within the context of the alien society (i.e., “In this
alien society, how much potential value does the artwork above have?”). Half of the additional
paintings shown to each participant were by the same artist as the prototype. These paintings
were of a similar style to the prototype and therefore lower novelty within the context. The other
half of the paintings shown to each participant were by a second artist. These paintings were of a
different style than the prototype and therefore higher novelty within the context.
We also varied, across participants, which of the two artists’ paintings was used as the
prototype painting. As such, while the exact same paintings were rated by all participants, half of
which were by one artist (“Artist A”) and half of which were by another artist (“Artist B”), we
experimentally varied whether a given painting was high vs. low novelty within the alien society
by randomly assigning participants to view only one of the two artists’ work as the prototype
(either a painting by Artist A or a painting by Artist B). By asking participants to each rate
paintings by both Artist A and Artist B, regardless of which artist’s work they saw as the

NOVELTY AND VARIABILITY 10
prototype, we were able to examine whether there was more variability in judgments of identical
paintings when they were construed as higher novelty within a given context compared to when
they were construed as lower novelty within a given context.
As predicted, there was more variability in value judgments for the high novelty paintings
(n = 80 painting stimuli; MSD = 2.07, SD = 0.22) than the low novelty paintings (n = 80 painting
stimuli; MSD = 1.79, SD = 0.26), t(158) = 7.45, p < .001, d = 1.18, 95% CI for difference [.21,
.36]. This effect was robust in the condition in which Artist’s A’s work was the prototype (n = 40
painting stimuli), F(1, 156) = 17.22, p < .001, ηp2 = .01, 95% CI for difference [.11, .29], and in
the condition in which Artist B’s work was the prototype (n = 40 painting stimuli), F(1, 156) =
57.06, p < .001, ηp2 = .27, 95% CI for difference [.27, .46]), with a larger effect in the condition
in which Artist B’s work was the prototype, F(1, 156) = 5.80, p = .017, ηp2 = .04.
Study 4: Mechanistic Evidence. Studies 1-3 showed that, as ideas become more novel,
there is more variability in how evaluators judge their value. Study 4 examined a boundary
condition: whether the positive relationship between novelty and variability might diminish
when the value of an idea was closely tied to it how novel it was. Explicitly construing value as
dependent on novelty might provide observers with clear criteria on which to judge the value of
unfamiliar ideas, reducing variability in value judgments for higher novelty ideas. Construing
value in terms of novelty might also limit the applicability of existing templates for assessing
idea value, as the evaluative context is unfamiliar, rendering value judgments more difficult and
susceptible to idiosyncratic variability regardless of idea novelty28. Both lines of reasoning led us
to expect that the effect observed thus far—greater variability in value judgments of more novel
ideas—would be attenuated when value was explicitly defined in terms of novelty.

NOVELTY AND VARIABILITY 11
We examined this prediction in the context of ideas for sandwiches. First, we conducted a
pretest in which we recruited 101 U.S. residents from Prolific with experience working in the
hospitality industry, providing a sample with some domain knowledge. We asked participants to
rate the novelty of 40 ideas for sandwiches. We then selected the 19 sandwich ideas that were
judged to be higher novelty (significantly higher than the scale mid-point) and the 19 sandwich
ideas that were judged to be lower novelty (significantly lower than the scale mid-point) to use in
our main study. For pre-test details, see Method.
Next, we recruited a separate sample of 200 U.S. residents with experience working in
the hospitality industry to rate the value offered by the 38 sandwich ideas identified in the
pretest, half of which were higher novelty and half of which were lower novelty. Participants
were randomly assigned to either a baseline condition or a novelty-as-value condition in which
we defined value as dependent on novelty. Specifically, in the baseline condition, for each
sandwich idea, participants were asked, “How successful would this sandwich be as a menu
item?” In the novelty-as-value condition, for each sandwich idea, participants were asked, “At a
restaurant specializing in sandwiches no one has tried before, how successful would this
sandwich be as a menu item?”.
In our pre-registered analysis plan, we indicated that we would analyze the data using
ANOVA to test for an interaction between condition (baseline vs. novelty-as-value) x idea
novelty (low vs. high) on average standard deviation in value ratings, followed by mean
comparisons to test our predictions. However, the data did not meet the assumption of equality of
variances, so we analyzed the data using ordinary least squares (OLS) regression with Huber-
White robust standard errors to account for heteroscedasticity. We report all results using our

NOVELTY AND VARIABILITY 12
original, pre-registered analysis plan in the Online Supplement, which yielded results in line with
our predictions.
Figure 2: Value Standard Deviation as a Function of Condition (Study 4).
Consistent with our pre-registered predictions, there was an idea novelty (low vs. high) x
condition (baseline vs. novelty-as-value) interaction, b = -.58, SE = .07, t(72) = -8.13, p < .001,
95% CI [-.72,-.44] (see Figure 2). In the baseline condition (n = 38 ideas), there was more
variability in value judgments for the higher novelty ideas (n= 19 ideas; MSD = 1.59, SD = 0.13)
compared to the lower novelty ideas (n = 19 ideas; MSD = 1.21, SD = 0.22), b = .37, SE = .06,
t(72) = 6.11, p < .001, 95% CI for difference [.25, .50]. This effect reversed in the novelty-as-
value condition (n = 38 ideas), with the lower novelty ideas showing greater value variability (n
= 19 ideas; MSD = 2.06, SD = 0.09) than the higher novelty ideas (n = 19 ideas; MSD = 1.86, SD =
1.2
1.6
2.0
Baseline Novelty−as−value
Condition
Average Standard Deviation in Value Ratings
Lower novelty sandwich ideas Higher novelty sandwich ideas

NOVELTY AND VARIABILITY 13
0.12), b = -.21, SE = .04, t(72) = -5.63, p < .001, 95% CI for difference [-.27, -.13]. Although
not pre-registered, we also observed that variability was greater in the novelty-as-value condition
than in the baseline condition for both the low novelty ideas, b = .85, SE = .06, t(72) = 14.90, p <
.001, 95% CI for difference [.73, .96], and the high novelty ideas, b = .27, SE = .04, t(72) = 6.22,
p < .001, 95% CI for difference [.18,.35].
We found consistent results when we tested for an interaction between idea novelty score
(generated in our pretest) and condition (baseline vs. novelty as value) on value standard
deviation (see Online Supplement). Our predictions were also supported when we analyzed only
responses from participants who indicated they had experience working in restaurants
specifically (n = 142 participants or 71.0% of the sample, see Online Supplement).
Study 5: Impact on Intent to Invest. Studies 1-4 demonstrated more variability in
judgments of the value of higher novelty (vs. lower novelty) ideas. In Study 5, we examined one
consequence of this effect. We theorized that more variability in value judgments for higher
novelty ideas might present a barrier for these ideas because variability might signal risk,
diminishing evaluators’ willingness to support newer ideas.
We recruited 401 U.S. residents with investment experience from Prolific. Participants
were asked to consider whether they would invest in the following product idea: “A device that
directs sound from tablet/laptop speakers to the user, works by funneling the sound from the
speakers back in the direction of the user.” Participants were then asked to imagine they had
consulted experts, colleagues, and others they trusted for advice, asking them for feedback on
how valuable they thought the investment opportunity was. Participants then read that each
person had given the idea a value rating ranging from 1 star (worst rating) to 5 stars (best rating).
Participants were then randomly assigned to one of two conditions (high variability vs. low

NOVELTY AND VARIABILITY 14
variability). Participants in the high variability condition saw a graph showing a higher
variability of ratings; participants in the low variability condition saw a graph showing a lower
variability of ratings (see Figure 3). The mean rating across conditions was identical (3.0).
Participants then completed three items capturing their intent to invest in the idea. Next,
participants rated, in a randomized order, how risky the idea was and how novel the idea was.
For details on exact items used, see Methods. Finally, as a manipulation check, participants
indicated how similar the idea ratings were.
Figure 3: Manipulation Stimuli (Study 5).
In our pre-registered analysis plan, we indicated that we would analyze the data using
independent samples t-tests to examine the effect of condition (low variability vs. high
variability) on our dependent measures. However, the data did not meet the assumption of
normality, so we analyzed the data using non-parametric Mann-Whitney U tests instead of t-
tests. We report all results using our original, pre-registered analysis plan in the Online
Supplement, which yielded results in line with our predictions.
Our manipulation check confirmed that perceived similarity of ratings was higher in the
low variability condition (n =200 participants) (M = 4.29, SD = 1.34) than in the high variability

NOVELTY AND VARIABILITY 15
condition (n = 201 participants) (M = 3.97, SD = 1.49), z(1,399) = -2.29, p = .022, Cliff ’s delta =
.13, 95% CI for the delta estimate = [.02,.34]. Consistent with predictions, participants in the
high variability condition indicated lower intent to invest in the idea (M = 2.51, SD = 1.01)
compared to participants in the low variability condition (M = 2.96, SD = 1.13), z(1,399) =
−4.13, p < .001, Cliff ’s delta = .24, 95% CI for the delta estimate = [.13,.24]). Participants also
judged the investment to be riskier in the high variability condition (M = 5.61, SD = 1.09)
compared to the low variability condition (M = 5.24, SD = 1.15), z(1,399) = -3.52, p < .001, Cliff
’s delta = -.20, 95% CI for the delta estimate = [-.31, -.09]. The effect of condition on ratings of
idea novelty was non-significant, Mhigh = 3.98, SD = 1.51, Mlow = 3.78, SD = 1.46), z(1,399) = -
1.37, p = .169, Cliff ’s delta = .08, 95% CI for the delta estimate = [-.19, .03], indicating that our
manipulation of value judgment variability did not significantly impact perceptions of how novel
the idea was. There was an indirect effect of condition on intent to invest through perceived risk,
b = -.12, SE = .05, 95% CI [-.23, -.04], indicating that one reason that greater variability in
judgments of an idea’s value leads to lower intent to invest in the idea is that greater variability
signals greater risk (see Figure 4).
Figure 4: Mediational Model (Study 5)

NOVELTY AND VARIABILITY 16
Discussion
Innovation is a goal prioritized across the arts, sciences, and business3,12,14. Despite this
general desire for novel products and solutions, our research suggests that when evaluators
actually encounter novel ideas—the very ideas society hopes to advance—these ideas may tend
to generate disagreement. Specifically, we showed that judgments of the value of highly novel
ideas are more varied across evaluators than judgments of less novel ideas. We found this effect
in multiple contexts, using archival and experimental methods, and in convenience samples and
samples of evaluators with domain knowledge. Furthermore, we found that people interpret
variability in value judgments as a negative signal, reducing their support for novel ideas. Our
results, particularly those of Study 4, suggest that evaluators’ judgments of novel ideas may vary
because evaluators have relatively few common templates against which to evaluate them,
making judgments more reliant on idiosyncratic knowledge and preferences. Broadly, our
findings are practically important because they highlight that the way human minds, in
aggregate, process novelty may be at odds with society’s goal to innovate. Our findings also
offer practical insight into why creative ideas often generate conflict and may seem socially
controversial23,45,46,47,8.
Our research contributes to the literature on evaluation of creative ideas. Previous studies
point to a bias against novel ideas, or a tendency to for evaluators to judge highly novel ideas
more negatively on average than less novel ones7,8. Our findings build on this work by showing
that more novel ideas may not only elicit negative reactions, but may also produce greater
variability in evaluators’ reactions, advancing our understanding of the psychological processes
at play in creative idea evaluation. In Study 5, we also found that greater variability in value
judgments (compared to less variability) reduces people’s intent to invest in ideas, even when

NOVELTY AND VARIABILITY 17
average judged value was held constant. This result points to a second-order effect (i.e., people’s
judgments about variability in others’ judgments) that may operate as an independent
mechanism, alongside a general negativity toward novel ideas, explaining why creative ideas
often lose momentum in the social world4,5,6.
Our findings advance the perspective within the creativity literature that assessments of
creative ideas may be inconsistent across different sets of judges. Previous studies demonstrated
that ratings of the “creativity” of ideas may vary between groups (for instance, creator ratings vs.
peer ratings), calling into question the reliability of creativity judgments18,21,48,49. Our research,
particularly Study 3’s evidence for the causal role of novelty in impacting variability in value
judgments, builds on these findings and offers evidence that judgments may become more
inconsistent between individual evaluators as ideas become more creative. Our findings are
notable given that we identified our predicted effect in domains often examined in creativity
research, such as entrepreneurial pitches, films, and visual art18,19,50,51,52. Our findings are also
broadly consistent with previous theoretical work in the creativity literature questioning whether
judgments of the “value” of creative ideas might be prone to error and may vary depending on
context53.
Our research also has implications for decision making more broadly by furthering our
understanding of the factors that drive consensus, or its inverse, judgmental variability28,54.
Recent work has drawn attention to how irrelevant individual and contextual factors, such as a
person’s mood or the weather, produce unwanted inconsistency across decision-makers28. We
build on this work by showing that evaluators’ judgments may be especially prone to variability
when the stimuli being evaluated are new. Our studies demonstrate that consensus is a feature of
the familiar—novelty itself produces judgmental variability.

NOVELTY AND VARIABILITY 18
Our research has several notable limitations. Our studies captured idea value using
subjective ratings on a Likert-type scale. Future research is needed to test whether our effects
extend to other measures of value, such as numerical assessments of an idea’s potential value in
terms of revenue to an organization over a certain period of time. The majority of our studies
also captured idea novelty using subjective assessments. This approach was aligned with our
theoretical account, as we hypothesized that ideas that subjectively seem more novel should also
produce more value judgment variability. Study 3’s results also showed that our effects extended
to a context in which we objectively manipulated novelty (by varying the extent to which a focal
stimulus was similar to/different from a prototype), providing some evidence that our effects are
generalizable to situations beyond those in which novelty is subjectively assessed. Still, more
research is needed to extend our findings using objective measures of novelty45,55,56.
The evidence we provide of our mechanistic account is also limited. We theorized that a
lack of common templates could explain greater variability in judgments of idea value as ideas
become more novel. The empirical evidence we present offers some evidence consistent with
this account, showing that our predicted effect is diminished when we offer evaluators a common
template (idea novelty) on which to assess idea value (Study 4). However, more research is
needed to rule in our theoretical mechanism. For instance, future studies could directly measure
whether evaluators increasingly call to mind different templates and different evaluative criteria
as ideas become more novel.
Several potential boundary conditions could also be explored in future research. Our
studies examined judgments of idea value made by convenience samples and samples of
evaluators with domain knowledge (i.e., Study 2 which sampled audience members at Sundance
and Study 4 which sampled evaluators with experience working in the hospitality industry). Our

NOVELTY AND VARIABILITY 19
results thus suggest that our predicted effects extend to samples of evaluators with some degree
of expertise. It is possible, however, that we could observe relative consensus in judgments of the
value of novel ideas in samples of experts, as experts may be more likely to share the same
evaluative criteria21,49,57. The Consensual Assessment Technique16, a commonly used method for
scoring the creativity of ideas, in which creativity scores are aggregated across judges, is
recommended to be used specifically with experts for this reason. The possibility of moderation
of our predicted effects by evaluator expertise is consistent with the results of Study 4, in which
we found that providing judges with shared evaluative criteria diminished our effect.
Relatedly, Study 4’s results suggest that in domains in which evaluators view the novelty
of a particular idea as synonymous with its value, or value is defined in a highly specific way, the
positive relationship between novelty and variability may diminish. Our studies also only
examined contexts in which evaluators' value judgments were independent. The novelty-
variability relationship may be less likely to emerge when evaluators are aware of others’
assessments, as anchoring effects and conformity pressures may drive consensus in judgments58.
There may also be a curvilinear effect of novelty on variability. While the empirical scope of our
studies did not provide enough exceptionally novel ideas to test this possibility, future studies
could examine whether, at very high levels of novelty, opinions of value might converge again,
as has been argued for assessments of “unicorn” ideas in the technology sector59.
Future research could also explore additional consequences of variability in judgments of
novel ideas, beyond investment intentions. For instance, future studies could explore whether
people who seek others’ opinions about their own ideas are less likely to pursue creative ideas
because of variability in feedback2,41. We also specifically examined the consequences of
variability when the distribution of value ratings was centered at the midpoint on the scale.

NOVELTY AND VARIABILITY 20
Future research could test how varying levels of variability in different distributions (positive vs.
negative skewed distributions) impacts evaluators’ support for ideas.
Conclusion
New ideas are an important source of innovation. This research identifies one reason why
the implementation of creative ideas might become stalled—evaluators disagree more about the
value of more novel ideas than less novel ideas. Observing less consensus (i.e., more variability)
in others’ judgments of the value of an idea can subsequently diminish people’s interest in
supporting creative ideas, as disagreement is seen as a sign of risk. Our findings suggest that, for
those seeking new ideas, a certain amount of disagreement about the value of a creative idea may
be expected, as we find that disagreement is a byproduct of novelty.
Methods
All studies received ethics approval from the institutional review board at Cornell
University (Protocols #2102010124 and #1905008829) and comply with all relevant ethical
regulations. For all studies except for Study 2 (analysis of archival data), study design, sample
size, predictions, exclusion criteria, and analysis plans were pre-registered at AsPredicted.org.
Studies 1, 3, 4, and 5 were preregistered on April 6, 2022, August 14, 2022, December 29, 2021,
and April 20, 2022 respectively. The data for Studies 1, 3, 4, and 5 were collected using
Qualtrics. Informed consent was obtained from all study participants in Studies 1, 3, 4, and 5.
Participants in Studies 1, 4, and 5 were compensated for their time with a flat fee (Study 1: $0.20
USD; Study 4: $0.80 USD; Study 5: $0.45 USD). For Study 3 participants, the researchers paid
Lucid Theorem $1.00 USD per participant. Regarding participant compensation, Lucid
Theorem’s website provides the following information, “Our respondents are sourced from a
variety of supplier types who have control over incentivizing their respondents based on their

NOVELTY AND VARIABILITY 21
business rules…Some suppliers do not incentivize their respondents at all, most provide loyalty
reward points or gift cards, and some provide cash payments”60.
For all studies using the same sampling source, participants who took part in one study
were excluded from all subsequent studies. Data collection and analysis were not performed
blind to the conditions of the experiments. For all studies, anonymized data, code, and materials,
including all idea stimuli used, are available on the Open Science Framework (OSF) at
https://osf.io/h3puf/?view_only=f042b664a538496887a30e9e791fbca9.
Study 1
Pretest. We gathered descriptions of the 1088 ventures pitched on seasons 1-12 of Shark
Tank from https://allsharktankproducts.com (all seasons available at the time of the study). We
recruited a sample of 1927 U.S. residents from Prolific Academic and Mechanical Turk (Mturk)
(Mage = 38.41 , SD = 28.69; 51.9% women, 46.7% men; 1.5% other gender identity) to rate the
novelty of each pitch on a 7-point scale (1= not at all novel, 7 = extremely novel). Novelty was
defined as the degree to which the pitch was unusual, unique, or unfamiliar16. Each participant
rated a random subset of 10-20 pitches, providing approximately 30 ratings per pitch. We
selected the 250 pitches rated most novel and the 250 pitches rated least novel to use in our main
study. For exact descriptions of all pitches used in the main study, see the OSF page. A
sensitivity analysis using G*Power61 indicated that, at α = .05, our sample size of 500 pitches
provided 80% power to detect a minimum effect of d = 0.25.
Main Study. The study pre-registration is available at
https://aspredicted.org/blind.php?x=76Z_274. The study was launched on December 21, 2022.
We recruited a sample of 1000 U.S residents from Mturk (Mage = 33.25, SD = 9.79; 49% women,
50% men; 1% other gender identity). No participants were excluded from analyses. Participants

NOVELTY AND VARIABILITY 22
rated the value of a random subset of 15 of the 500 pitches identified in the pretest, providing
approximately 30 ratings per pitch. For each idea, participants were asked, “What is the degree
of value offered by this idea?” rated on a 7-point scale (1 = extremely low value, 7 = extremely
high value).
Study 2
Film Novelty (Independent Variable). Film makers can submit their feature length films
for consideration in one of four categories at the Sundance film festival: U.S. Documentary,
World Documentary, U.S. Narrative, or World Narrative. Films considered for these categories
are also considered by festival programmers for two additional categories: NEXT and Midnight.
NEXT features, “Pure, bold works distinguished by an innovative approach to storytelling,”
while Midnight features, “An eclectic mix of horror, sci-fi, over-the-top comedy, explicit
animation, and bizarre stories that defy categorization”62. Hence, all feature length films
submitted to Sundance are either rejected or are sorted by programmers into one of two groups:
1) films accepted into the original category submitted to (U.S. Documentary, World
Documentary, U.S. Narrative, World Narrative), which we labeled lower novelty categories, or
2) films accepted into NEXT or Midnight, which we labeled the higher novelty categories. We
confirmed through a former programmer at Sundance who is a personal contact of one of the
authors that this characterization of the relative novelty of films in the six feature film categories
is accurate.
Audience Ratings Standard Deviation (Outcome Measure). We used Sundance archival
audience ratings of the films as a proxy for value, which we collected from the website
https://cannes-ratings.tk/Sundance on August 16, 2022. This website automatically standardized
each film rating collected into a 10-point scale and provided the standard deviation of ratings for

NOVELTY AND VARIABILITY 23
each film, which we used as our dependent measure. Of the 539 films listed on a website, 16
films had one rating; thus a standard deviation could not be computed, leaving 523 films in our
analyses.
Control Variable. Number of ratings varied across films. Given that number of ratings
impacts ratings standard deviation, we included numbers of ratings as a control in our analyses.
Study 3
The study pre-registration is available at https://aspredicted.org/blind.php?x=89W_9GJ.
The study was launched on August 14, 2022. We recruited 200 U.S. residents from Lucid
Theorem, an online platform that provides nationally representative samples of U.S. residents
(Mage = 44.42, SD = 16.78; 51% women, 49% men)63. No participants were excluded from
analyses.
Participants were randomly assigned to one of two conditions. Participants in one
condition saw a painting by Artist A (Josef Albers) as the example of highly acclaimed art within
the alien society. Participants in the other condition saw a painting by Artist B (Fritz Winter) as
the example of highly acclaimed art within the alien society. Participants then judged the value
of a random subset of 20 paintings from a group of 80 paintings, half by Artist A and half by
Artist B. See OSF page for all painting stimuli used.
For each painting evaluated, participants were asked, “In this alien society, how much
potential value does the artwork above have?” rated on a 7-point scale (1= extremely low value, 7
= extremely high value). For participants who viewed Artist A’s work as the prototype, ratings of
Artist A’s paintings were coded as low novelty and ratings of Artist B’s paintings were coded as
high novelty. For participants who viewed Artist B’s work as the prototype, ratings of Artist B’s
paintings were coded as low novelty and ratings of Artist A’s paintings were coded as high

NOVELTY AND VARIABILITY 24
novelty. Across conditions, each of the 80 paintings were both shown as a low novelty painting
and as a high novelty painting; thus, our sample size consisted of 160 painting stimuli. A
sensitivity analysis using G*Power indicated that, at α = .05, this sample size provided 80%
power to detect a minimum effect of d = .45.
Study 4
Pretest. We recruited 101 U.S. residents from Prolific (Mage = 32.22, SD = 10.38; 50%
women, 48% men, 2% other). Using Study 1’s novelty measure, each participant rated the
novelty of a random subset of 10 sandwich ideas from a list of 40 sandwich ideas, providing 25
ratings per idea. We selected 19 ideas that were significantly below the scale midpoint (lower
novelty ideas) and 19 ideas that were significantly above the scale midpoint (higher novelty
ideas) to use in our main study.
Main Study. The study pre-registration is available at
https://aspredicted.org/blind.php?x=X84_T8Y. The study was launched on December 29, 2021.
We recruited 200 U.S. residents with experience working in the hospitality industry on Prolific
(48% women, 51% men, Mage = 36.01, SDage = 12.22). No participants were excluded from
analyses. Participants were randomly assigned to either a baseline condition or a novelty-as-
value condition in which we defined value as dependent on novelty. In both conditions,
participants each rated all 38 sandwich ideas identified in the pretest. Participants in the baseline
condition were asked, “How successful would this sandwich be as a menu item?” rated on a 7-
point scale (1 = not successful at all to 7 = extremely successful). Participants in the novelty-as-
value condition were asked, “At a restaurant specializing in sandwiches no one has tried before,
how successful would this sandwich be as a menu item?” rated on the same 7-point scale. At the
end of the survey, participants indicated whether they had experience working in restaurants

NOVELTY AND VARIABILITY 25
specifically. Each of the 38 sandwich ideas were rated both in the baseline condition and in the
novelty-as-value condition. A sensitivity analysis using G*Power indicated that, at α = .05, our
sample size of 76 ideas provided 80% power to detect a minimum effect of ηp2 = .10.
Study 5
The study pre-registration is available at https://aspredicted.org/blind.php?x=WJ5_5KL.
The study was launched on April 21, 2022. We recruited 401 U.S. residents with investment
experience from Prolific (Mage = 42.89, SD = 16.78; 50% women, 50% men; years of investment
experience: M = 9.30, SD = 9.35). No participants were excluded from analyses. A sensitivity
analysis using G*Power indicated that, at α = .05, our sample size of 401 participants provided
80% power to detect a minimum effect of d = 0.28.
Participants were asked to consider whether they would invest in the following product
idea: “A device that directs sound from tablet/laptop speakers to the user, works by funneling the
sound from the speakers back in the direction of the user.” Participants were then asked to
imagine they had consulted experts, colleagues, and others they trusted for advice, asking them
for feedback on how valuable they thought the investment opportunity was. Participants then
read that each person had given the idea a value rating ranging from 1 star (worst rating) to 5
stars (best rating). Participants were then randomly assigned to one of two conditions (high
variability vs. low variability). Participants in the high variability condition saw a graph showing
a higher variability of ratings; participants in the low variability condition saw a graph showing a
lower variability of ratings (see Figure 4). The mean rating across conditions was identical (3.0).
Participants completed three items capturing intent to invest: a) "I intend to invest in this
company, b) "Investing in this company will be profitable for me", and c) "Investing in this
company is a beneficial decision for me", rated on a 7-point scale (1 = strongly disagree, 7 =

NOVELTY AND VARIABILITY 26
strongly agree) (α = .92). Next, participants rated, in a randomized order, how risky and how
novel the idea was. Three items captured perceived risk: (a) "This idea is risky", b) "There is a
risk the idea will fail", c) "There is a risk this idea will not succeed" rated on a 7-point scale (1 =
strongly disagree, 7 = strongly agree) (α = .89). Three items captured perceived novelty: a) "This
idea is unusual", b) "This idea is novel", c) "This idea is unique", rated on a 7-point scale (1 =
strongly disagree, 7 = strongly agree) (α = .93). Finally, as a manipulation check, participants
indicated how similar the ratings of the idea were on a 7-point scale (1= the assessments of the
idea were very different, 7 = the assessments of the idea were very similar).
Data Availability Statement
De-identified participant data for all studies are permanently and publicly available on the Open
Science Framework at: https://osf.io/h3puf/?view_only=f042b664a538496887a30e9e791fbca9
Code Availability Statement

NOVELTY AND VARIABILITY 27
The code to replicate the analyses in the manuscript and our Online Supplement is available
permanently and publicly on the Open Science Framework at:
https://osf.io/h3puf/?view_only=f042b664a538496887a30e9e791fbca9
Acknowledgements
This research was supported by funds from the ILR School, Cornell University and an
Innovation, Entrepreneurship, and Technology theme grant from the Johnson College of
Business, Cornell University. The funders had no role in study design, data collection and
analysis, decision to publish or preparation of the manuscript. We are grateful to E. Mannix and
the members of ExPO Lab for their feedback on this research. We thank S. Parry for advice on
statistical analyses. We thank S. Owens for providing information about the film categories
at Sundance.
Author Contributions Statement
W.J. developed the study concept. W.J. and D.P. designed the studies. W.J. and D.P. collected
and analyzed the data. W.J. drafted the manuscript. D.P. revised the manuscript and prepared the
final text for submission.
Competing Interests Statement
The authors declare no competing interests.

NOVELTY AND VARIABILITY 28
Table 1
Linear Regressions Predicting Ratings Standard Deviation (Study 2)
Model 1
Model 2
Model 3
(n ratings > 19)
NEXT (reference category)
b(SE)
t(517)
p
95% CI
t(516)
p
95% CI
b(SE)
t(319)
p
95% CI
U.S. Documentary
-.323
(.06)
-5.34
<.001
[-.44, -.20]
-.336
(.06)
-5.51
<.001
[-.46, -.22]
-.359
(.05)
-7.56
<.001
[-.45, -.27]
World Documentary
-.330
(.06)
-5.21
<.001
[-.45, -.21]
-.350
(.06)
-5.47
<.001
[-.48, -.22]
-.318
(.05)
-5.92
<.001
[-.42, -.21]
U.S. Narrative
-.188
(.06)
-3.33
.001
[-.30, -.08]
-.167
(.06)
-3.00
.003
[-.28, -.06]
-.202
(.05)
-4.11
<.001
[-.30, -.11]
World Narrative
-.170
(.06)
-2.87
.004
[-.29, -.05]
-.174
(.06)
-2.93
.004
[-.29, -.06]
-.143
(.06)
-2.59
.010
[-.25, -.03]
Midnight
-.014
(.08)
-0.17
.867
[-.17, .15]
-.005
(.08)
-0.06
.949
[-.16, .15]
-.030
(.08)
-0.38
.706
[-.19, -.13]
Number of ratings
-.15Æ
(.00)
-4.29
<.001
Midnight (reference category)
U.S. Documentary
-.310
(.08)
-3.84
<.001
[-.47, -.15]
-.330
(.08)
-4.12
<.001
[-.49, -.17]
-.328
(.08)
-4.27
<.001
[-.48, -.18]
World Documentary
-.317
(.08)
-3.83
<.001
[-.48, -.15]
-.345
(.08)
-4.17
<.001
[-.51, -.18]
-.288
(.08)
-3.56
<.001
[-.45, -.13]
U.S. Narrative
-.175
(.08)
-2.25
.025
[-.33, -.02]
-.162
(.08)
-2.13
.033
[-.31, -.01]
-.172
(.08)
-2.21
.028
[-.33, -.02]
World Narrative
-.157
(.08)
-1.97
.050
[-.31, .00]
-.169
(.08)
-2.14
.033
[-.32, -.01]
-.113
(.08)
-1.38
.169
[-.27, .05]
NEXT
.014
(.08)
0.17
.867
[-.15, .17]
.005
(.08)
0.64
.949
[-.15, .16]
.030
(.08)
0.38
.706
[-.13, .19]
Number of ratings
-.152Æ
(.00)
-4.29
R2
.084
.106
.144
Note.ÆStandardized regression coefficient displayed. Unstandardized regression coefficients are displayed unless otherwise noted with
Huber-White robust standard errors in parentheses.

NOVELTY AND VARIABILITY
Figures Legends/Captions
Fig.1. Average Standard Deviation in Film Ratings a Function of Film Category (Study 2).
Films were the unit of analysis. The y axis represents the mean standard deviation in ratings for
each film. N U.S. Documentary = 107, N World Documentary = 80, N U.S. Narrative = 116, N
World Narrative = 85, N Midnight = 61, N NEXT = 74. Error bars represent bootstrapped 95%
confidence intervals.
Fig.2. Value Standard Deviation as a Function of Condition (Study 4). Sandwich ideas were
the unit of analysis. The y axis represents the mean standard deviation in ratings for each
sandwich idea. N Baseline condition, lower novelty sandwich ideas = 19, N Baseline condition,
higher novelty sandwich ideas = 19, N Novelty-as-value condition, lower novelty sandwich ideas
= 19, N Novelty-as-value condition, higher novelty sandwich ideas = 19. Error bars represent
bootstrapped 95% confidence intervals.
Fig.3. Manipulation Stimuli (Study 5).
Fig.4. Mediational Model (Study 5).
We used the PROCESS macro for SPSS to conduct a simple mediation model to test for the
indirect effect of condition (low variability in idea evaluations vs. high variability in idea
evaluations) on intent to invest in the idea through perceived risk. N low variability in idea
evaluations = 200; N high variability in idea evaluations = 201. Regression coefficients shown
are unstandardized. All tests are two-sided. Adjustments were not made for multiple
comparisons. a estimates the effect of condition on perceived risk: coefficient = .376, SE = .22,
t(399) = 3.35, p < .001, 95% CI [.16,.60], b estimates the effect of perceived risk on intent to
invest, holding condition constant: coefficient = -.323, SE = .05, t(399) = -7.15 , p < .001, 95%
CI [-.41, -.23]; c estimates the total effect of condition on intent to invest: coefficient = -.454, SE

NOVELTY AND VARIABILITY
= .11, t(399) = -4.23, p < .001, 95% CI [-.66, -.24]; c’ estimates the effect of condition on intent
to invest holding perceived risk constant: coefficient = -.332, SE = .10, t(399) = -3.24, p = .001,
95% CI [-.53, -.13].
1 Baer, M. (2012). Putting creativity to work: The implementation of creative ideas in
organizations. Academy of Management Journal, 55, 1102-1119.
2 Perry-Smith, J. E., & Mannucci, P. V. (2017). From creativity to innovation: The social
network drivers of the four phases of the idea journey. Academy of Management Review,
42, 53-79.
3 Berg, J. M. (2016). Balancing on the creative highwire: Forecasting the success of novel ideas
in organizations. Administrative Science Quarterly, 61, 433-468.
4 Staw B. M. (1995). Why no one really wants creativity. In Ford C. M., Gioia D. A. (Eds.),
Creative action in organizations: Ivory tower vision and real world voices (pp. 161-66).
Thousand Oaks, CA: Sage.

NOVELTY AND VARIABILITY
5 Zhou, J., Wang, X. M., Bavato, D., Tasselli, S., & Wu, J. (2019). Understanding the receiving
side of creativity: A multidisciplinary review and implications for management research.
Journal of Management, 45(6), 2570-2595.
6 Harvey, S., & Mueller, J. S. (2021). Staying alive: Toward a diverging consensus model of
overcoming a bias against novelty in groups. Organization Science, 32(2), 293-314.
7 Mueller, J. S., Melwani, S., & Goncalo, J. A. (2012). The bias against creativity: Why people
desire but reject creative ideas. Psychological Science, 23, 13-17.
8 Mueller, J., Melwani, S., Loewenstein, J., & Deal, J. J. (2018). Reframing the decision-makers’
dilemma: Towards a social context model of creative idea recognition. Academy of
Management Journal, 61(1), 94-110.
9 Lu, S., Bartol, K. M., Venkataramani, V., Zheng, X., & Liu, X. (2019). Pitching novel ideas to
the boss: The interactive effects of employees’ idea enactment and influence tactics on
creativity assessment and implementation. Academy of Management Journal, 62(2), 579-
606.
10 Acar, O. A., Tarakci, M., & Van Knippenberg, D. (2019). Creativity and innovation under
constraints: A cross-disciplinary integrative review. Journal of Management, 45(1), 96-
121.
11 Runco, M. A., & Jaeger, G. J. (2012). The standard definition of creativity. Creativity
Research Journal, 24(1), 92-96.
12 Boudreau, K. J., Guinan, E. C., Lakhani, K. R., & Riedl, C. (2016). Looking across and
looking beyond the knowledge frontier: Intellectual distance, novelty, and resource
allocation in science. Management Science, 62(10), 2765-2783.

NOVELTY AND VARIABILITY
13 Rindova, V. P., & Petkova, A. P. (2007). When is a new thing a good thing? Technological
change, product form design, and perceptions of value for product innovations.
Organization Science, 18, 217-232.
14 Criscuolo, P., Dahlander, L., Grohsjean, T., & Salter, A. (2017). Evaluating novelty: The role
of panels in the selection of R&D projects. Academy of Management Journal, 60(2), 433-
460.
15 Baldwin, C. Y., & Clark, K. B. (2000). Design rules: The power of modularity (Vol. 1). MIT
press.
16 Amabile, T. M. (1996). Creativity in context. Boulder, CO: Westview Press.
17 Kerr, N. L., & Tindale, R. S. (2004). Group performance and decision making. Annual Review
of Psychology, 55, 623-655.
18 Runco, M. A., McCarthy, K. A., & Svenson, E. (1994). Judgments of the creativity of artwork
from students and professional artists. The Journal of Psychology, 128(1), 23-31.
19 Kozbelt, A., Seidel, A., ElBassiouny, A., Mark, Y., & Owen, D. R. (2010). Visual selection
contributes to artists' advantages in realistic drawing. Psychology of Aesthetics,
Creativity, and the Arts, 4(2), 93–102.
20 Hickey, M. (2001). An application of Amabile's Consensual Assessment Technique for rating
the creativity of children's musical compositions. Journal of Research in Music
Education , 49 , 234 – 244 .
21 Kaufman, J. C., Baer, J., & Cole, J. C. (2009). Expertise, domains, and the consensual
assessment technique. The Journal of Creative Behavior, 43(4), 223-233.

NOVELTY AND VARIABILITY
22 Harvey, S., & Kou, C. Y. (2013). Collective engagement in creative tasks: The role of
evaluation in the creative process in groups. Administrative science quarterly, 58(3), 346-
386.
23 Khessina, O. M., Goncalo, J. A., & Krause, V. (2018). It’s time to sober up: The direct costs,
side effects and long-term consequences of creativity and innovation. Research in
Organizational Behavior, 38, 107-135.
24 Csikszentmihalyi, M. (1999). 16 implications of a systems perspective for the study of
creativity. Handbook of Creativity, 313.
25 Meyer, J. W., & Rowan, B. (1977). Institutionalized organizations: Formal structure as myth
and ceremony. American Journal of Sociology, 83, 340-363.
26 DiMaggio, P. (1997). Culture and cognition. Annual Review of Sociology, 23.
27 Moreau, C. P., Lehmann, D. R., & Markman, A. B. (2001). Entrenched knowledge structures
and consumer response to new products. Journal of Marketing Research, 38, 14-29.
28 Kahneman, D., Sibony, O., & Sunstein, C. R. (2022). Noise. HarperCollins UK.
29 Amabile, T. M. (1988). A model of creativity and innovation in organizations. Research in
Organizational Behavior, 10, 123-167.
30 Miron-Spektor, E., & Erez, M. (2017). Looking at creativity through a paradox lens. The
oxford handbook of organizational paradox, 434-451.
31 Mueller, J., & Yin, Y. (2021). Unraveling the bias against novelty: guiding the study of our
tendency to desire but reject the new. Handbook of Research on Creativity and
Innovation, 267-289.
32 Fleming, L. (2001). Recombinant uncertainty in technological search. Management Science,
47, 117-132.

NOVELTY AND VARIABILITY
33 Mumford, M. D., Blair, C., Dailey, L., Leritz, L. E., & Osburn, H. K. (2006). Errors in creative
thought? Cognitive biases in a complex processing activity. The Journal of Creative
Behavior, 40(2), 75-109.
34 Licuanan, B. F., Dailey, L. R., & Mumford, M. D. (2007). Idea evaluation: Error in evaluating
highly original ideas. The Journal of Creative Behavior, 41(1), 1-27.
35 Elsbach, K. D., & Kramer, R. M. (2003). Assessing creativity in Hollywood pitch meetings:
Evidence for a dual-process model of creativity judgments. Academy of Management
Journal, 46, 283-301.
36 Goncalo, J. A., Flynn, F. J., & Kim, S. H. (2010). Are two narcissists better than one? The link
between narcissism, perceived creativity, and creative performance. Personality and
Social Psychology Bulletin, 36(11), 1484-1495.
37 Mueller, J. S., Wakslak, C. J., & Krishnan, V. (2014). Construing creativity: The how and why
of recognizing creative ideas. Journal of Experimental Social Psychology, 51, 81-87.
38 Proudfoot, D., & Fath, S. (2021). Signaling creative genius: How perceived social
connectedness influences judgments of creative potential. Personality and Social
Psychology Bulletin, 47(4), 580-592.
39 Zhou, J., Wang, X. M., Bavato, D., Tasselli, S., & Wu, J. (2019). Understanding the receiving
side of creativity: A multidisciplinary review and implications for management research.
Journal of Management, 45(6), 2570-2595.
40 Hunter, S. T., Blocker, L. D., Gutworth, M. B., & Allen, J. (2022). Why we Support some
Original Ideas but Reject Others: An Application of Signaling Theory. The Journal of
Creative Behavior.

NOVELTY AND VARIABILITY
41 Harrison, S. H., & Rouse, E. D. (2015). An inductive study of feedback interactions over the
course of creative projects. Academy of management Journal, 58(2), 375-404.
42 Dow, J., & da Costa Werlang, S. R. (1992). Uncertainty aversion, risk aversion, and the
optimal choice of portfolio. Econometrica: Journal of the Econometric Society, 197-204.
43 Whalen, J., Gallistel, C. R., & Gelman, R. (1999). Nonverbal counting in humans: The
psychophysics of number representation. Psychological Science, 10, 130-137.
44 Craig, B. (2016). Sundance - A Festival Virgin's Guide (3rd Edition): Surviving and thriving in
Park City at America's most important film festival. London: Cinemagine Media
Publishing.
45 Goncalo, J. A., & Staw, B. M. (2006). Individualism–collectivism and group creativity.
Organizational behavior and human decision processes, 100(1), 96-109.
46 Nemeth, C. J. (1986). Differential contributions of majority and minority influence.
Psychological review, 93(1), 23.
47 Nemeth, C. J., & Staw, B. M. (1989). The tradeoffs of social control and innovation in groups
and organizations. In Advances in experimental social psychology (Vol. 22, pp. 175-210).
Academic Press.
48 Runco, M. A., & Smith, W. R. (1992). Interpersonal and intrapersonal evaluations of creative
ideas. Personality and Individual Differences, 13(3), 295-302.
49 Kaufman , J. C. , Baer , J. , Cole , J. C. , & Sexton , J. D. (2008). A comparison of expert and
nonexpert raters using the Consensual Assessment Technique. Creativity Research
Journal, 20, 171 – 178 .

NOVELTY AND VARIABILITY
50 Berg, J. M., & Yu, A. (2021). Getting the picture too late: Handoffs and the effectiveness of
idea implementation in creative work. Academy of Management Journal, 64(4), 1191-
1212.
51 Simonton, D. K. (2009). Cinematic success criteria and their predictors: The art and business
of the film industry. Psychology & marketing, 26(5), 400-420.
52 Davis, B. C., Hmieleski, K. M., Webb, J. W., & Coombs, J. E. (2017). Funders' positive
affective reactions to entrepreneurs' crowdfunding pitches: The influence of perceived
product creativity and entrepreneurial passion. Journal of business venturing, 32(1), 90-
106.
53 Weisberg, R. W. (2015). On the usefulness of “value” in the definition of creativity. Creativity
research journal, 27(2), 111-124.
54 Lee, C. Y., & Morewedge, C. K. (2022). Noise Increases Anchoring Effects. Psychological
Science, 33, 60-75.
55 Ward, T. B., Patterson, M. J., Sifonis, C. M., Dodds, R. A., & Saunders, K. N. (2002). The role
of graded category structure in imaginative thought. Memory & Cognition, 30, 199-216.
56 Maio, S., Dumas, D., Organisciak, P., & Runco, M. (2020). Is the reliability of objective
originality scores confounded by elaboration?. Creativity Research Journal, 32(3), 201-
205.
57 Baer, J., & McKool, S. S. (2009). Assessing creativity using the consensual assessment
technique. In Handbook of research on assessment technologies, methods, and
applications in higher education (pp. 65-77). IGI Global.

NOVELTY AND VARIABILITY
58 Wilson, T. D., Houston, C. E., Etling, K. M., & Brekke, N. (1996). A new look at anchoring
effects: basic anchoring and its antecedents. Journal of Experimental Psychology:
General, 125, 387.
59 Lee, A. (2013). Welcome to the unicorn club: Learning from billion-dollar startups.
TechCrunch. Retrieved from https://techcrunch.com/2013/11/02/welcome-to-the-
unicorn-club/
60 Lucid Holdings Inc. (2022). Sampling Source FAQs. Retrieved from:
https://support.lucidhq.com/s/article/Sample-Sourcing-FAQs
61 Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* Power 3: A flexible statistical
power analysis program for the social, behavioral, and biomedical sciences. Behavior
Research Methods, 39(2), 175-191.
62 Sundance.org (2023). Sundance Film Festival 2023: Submitting your project. Pg. 4. Retrieved
from https://dotorg-stg-
assets.s3.amazonaws.com/pdf/submissions/2023_Submissions_FAQ.pdf
63 Coppock, A., & McClellan, O. A. (2019). Validating the demographic, political,
psychological, and experimental results obtained from a new source of online survey
respondents. Research & Politics, 6(1), 2053168018822174.