Content uploaded by Wagner A Kamakura
Author content
All content in this area was uploaded by Wagner A Kamakura
Content may be subject to copyright.
Reviewing the reviewers: The impact of individual film
critics on box office performance
Peter Boatwright & Suman Basuroy &
Wagner Kamakura
Received: 7 February 2006 /Accepted: 4 May 2007 /
Published online: 23 August 2007
#
Springer Science + Business Media, LLC 2007
Abstract Critics and their opinions or critical reviews play a major role in many
markets. Marketing resear ch on how critics impact product performance has so far
examined an aggregate critic eff ect. An obstacle in studies examining the
relationship of aggregate critical opinion and product sales is the close association
between the intrinsic quality of a product and the aggregate opinion regarding the
product. Our analysis parses out these two effects, allowing us to distinguish
individual critics who are simply good at identifying products with popular appeal
from those who act as opinion leaders and engender early product sales. The role of
critics is especially prominent in the film business, in which one finds multiple
expert opinions about each movie and where critics’ endorsements are used in
advertising. In the context of the motion picture industry, our research investigates
the imp act of individual film critics on the market performance of movies, where
specific key critics and reviewers may serve as market gatekeepers, and where
various critics may have different types of impacts on product performance.
Keywords Entertainment marketing
.
Motion picture distribution and exhibition
.
Movie choice
.
Predictors
.
Influencers
.
Wide-release
.
Platform-release
.
Movie critics
.
Stochastic variable selection
.
Bayesian models
.
New product research
JEL Classification C01
.
C11
.
C52
.
M31
Quant Market Econ (2007) 5:401–425
DOI 10.1007/s11129-007-9029-1
P. Boatwright (*)
Tepper School of Business, Carnegie Mellon University, Pittsburgh, PA 15213, USA
e-mail: pbhb@andrew.cmu.edu
S. Basuroy
Florida Atlantic University, Jupiter, FL 33458, USA
e-mail: sumanbasuroy@yahoo.com
W. Kamakura
Fuqua School of Business, Duke University, Box 90120, Durham, NC 27708, USA
e-mail: kamakura@duke.edu
1 Introduction
Critical and expert opinions provide im portan t and crucial i nform ation for
consumers in many “experience goods” markets—restaurants, shows/theaters,
books, movies, etc. (Basuroy et al. 2003; Caves 2000; Eliashberg and Shugan
1997; Greco 1997), because in these markets consumers cannot ascertain product
quality prior to actual consumption. For example, Reddy et al. (1998) point out that
Broadway show critics such as Frank Rich of the New York Times and Clive Barnes
of the New York Post had significantly greater influence on the longevity of the
shows they reviewed than did other critics. Even in markets in which rationality and
objectivity is expected to prevail, opinion leaders may play an important role in the
acceptance of new products. Stock analysts such as Henry Blodget of Merrill Lynch
and Anthony Noto of Goldman Sachs became household names and financial power
brokers (Regan 2001). Therefore, the opinion of experts is vital information in the
consumption of experience goods, and for such products the opinion of experts is
often sought. Our research assesses the specific role of individual expert opinions on
market performance after accounting for the effect of product quality.
The literature has inspected two separate roles of experts—influencers and
predictors (Basuroy et al. 2003; Eliashberg and Shugan 1997). Critics with
opinions that are correlated with early box office sales are defined as influencers,
while critics with opinions that are correlated with overall box office sales are
defined as predictors. With movies, for instance, the expert (critic) reveals the
stars, the director, the set s, and the plotline, in addition to his or her own overall
assessment of the film—details that can bring viewers to the theaters or cause them
to stay away. As such, a critic may serve the role of influencer, influencing early
box office sales by disseminating relevant information to the marketplace at the
moment the movie is released. On the other hand, the opinion of a critic expressed
at the outset of a movie may correlate with its overall box office sales. This
correlation could occur if a critic may anticipate a movie’s success or potential
market size, as has been observed in theatrical performances (Reddy et al. 1998).
Alternatively, the c orrelation could occur if a critic’s opinion serves as a bellwether
of audience tastes, for instance if the critic and audience have similar preferences.
As a label for correlation of critic opinion and overall box office performance,
Eliashberg and Shugan (1997)usedthetermpredictor (see, Basuroy et al. 2003;
Eliashberg and Shugan 1997 for d etails).
However, this literature and ap proach to assessing experts’ opinions and the
nature of their impact has a few major limitations. First and foremost, no empirical
study of the role of critical opinion in the marketing literature has accounted for the
intrinsic appeal of the experience good. Correlations between ticket sales and
aggregate critical acclaim found in the existing literature might be due to a third
factor—movie quality. Fo r example, it is entirely possible that good movies sell well
because they are good movies, liked by critics and by the public. Because of this
confounding factor of movie quality (or appeal), a positive correlation of aggregate
reviews and the weekly sales by itself may not reveal the role of critics. We believe
that this confound of movie quality raises questions regarding the interpretation of
the estimates previously obtained in the literature. Certainly, intrinsic appeal is
difficult to measure, because a host of facets of an artistic product join together for a
402 P. Boatwright et al.
Gestalt of excellence. Recent research by Kamakura et al. (2006) has provided a
measurement of the intrinsic appeal of an experience good after taking into account
rater biases and the differences in diagnosticity across critics. Rather than
summarizing critical opinion by an average or weighted average across critics,
Kamakura et al. (2006)’s technique uncovers and reports the latent dimension that
each of the critics are judging—namely, a movie’s appeal. Kamakura et al. (2006)’s
work underscores the fact that movie appeal and observed critical opinion are
correlated, in that critical opinions are themselves measures of movie appeal or
aspects of movie appeal. Its framework also implies that correlation of aggregate
critic opinion and movie appeal will in general be stronger than the correlation of an
individual critic’s opinion and movie appeal, because aggregate critic opinion
represents the consensus among the majority of experts.
In order to investigate the impact of critics after accounting for movie quality, our
model incorporates both the Kamakura et al. (2006) measure as well as opinions by
individual critics. As such, our estimates of the impact of individual critics show the
incremental impact of the reviews, after accounting for the combined movi e quality
and critic consensus. In other words, we measure how the unique contribut ions of
individual critics impact box office sales. There may be critics that do not offer any
unique insights beyond the readily apparent movie quality and critic consensus;
these critics without unique information will not appear to be incrementally
influential (with opinions correlated with early sales) or predictive (with opinions
correlated with overall sales). By including both the Kamakura et al. ( 2006) measure
and the opinions of individual critics, our model can assess the specific role of
critical opinion on market performance after accounting for the effect of movi e
quality or appeal on market performance.
Our use of individual critics not only allows us to ascertain the role of critics
separately from the contribution of movie quality but also yields richer detail on the
functioning of the market for experience goods. Eliashberg and Shugan (1997) and
others (Basuroy et al. 2003; Ravid 1999) assessed only the aggregate impact of
critics, without taking into account the confounding effect of movie quality, rather
than measuring the impact of individual critics. In an environment such as the film
industry and the financial world, in which certain well-known experts have much
greater media reach than others, the impact of the average expert (critic) may not
reflect the power of the well-known few. More generally, the nature of the impact of
critics may vary by critic—some may be quite powerful influencers, whereas others
may be good predictors. Since it is well known that in many industries individual
experts or critics play a significant part in shaping consumer behavior, we fill a gap
in this literature by using a methodology to identify and understand the incremental
role of an individual expert.
The use of individual opinions poses a significant modeling challenge, in that the
opinions of the numerous critics greatly expand the number of variables to be
incorporated into the model. To account for the large number of critics in the model,
we adopt a statistical technology that balances the numerous potential variables
with model parsimony. The modeling technology, developed by George and
McCulloch (1993) and known as stochastic search variable selection, is well known
in the statistics community but does not appear to have been utilized in the
mark eting literature to date.
Reviewing the reviewers: The impact of individual film critics 403
For the purpose of differentiating between influence and prediction, we utilize the
Bass model, the classic model about how consumers find out about a new product.
The Bass model is derived from the assumption that there are two sources of
information about a new product. One source of information is external to the pool
of consumers, such as advertising. The other source of information is internal to the
pool of consumers, such as word of mouth. The sales pattern over time of the new
product (box office sales of the movie) depends upon the strength of the impact of
the two sources of information. We consider movie critics as an external source of
information/persuasion, similar to advertising, that is made available immediately
before the movie is released. Therefore, we expect that if critics have influence as
defined above, their reviews would be related to the Bass model parameter
corresponding to the external source of information (the coefficient of innovation).
On the other hand, if critics correctly anticipate which movies will be successful or if
their opinions are bellwethers of market preferences, their reviews would be
correlated with market potential. It is possible that critics could both influence and
predict, in which case their reviews would be correlated with both of the
corresponding model parameters.
Our empirica l analysis of the role of individual critics and other factors on the
market performance of new movies produced three general types of findings. First,
our resul ts identify specific individual critics whose opinions correlate with the Bass
parameter that reflects the role of external information. For example, critics such as
Owen Gleiberman of Entertainment Weekly, Manohla Dargis of the LA Times,
Michael Wilmington of the Chicago Tribune, and Lawrence Van Gelder of the New
York Times wield noticeably more influence than several other critics (after
accounting for movie appeal) who have no discernable distinct impact on the
diffusion of a new movie. Second, our results show exactly which roles these critics
play—influencers or predictors. Although prior literature has concluded that critics
in aggregate act as predictors rather than influencers, we find certain individual
critics to be “influencers” (i.e., their views correlate with early adoption of the new
movie), but no individuals are identified as “predictors” (i.e., their views do not
correlate highly with the overall market potential). Third, in contrast to previous
generalizations for this product category, we find some effects to depend on the type
of movie. One examp le is the effect of advertising in the initial weeks of the movie’s
life cycle; we find that advertising is positively correlated with the coefficient of
innovation for “platform-release” movies (those released in a limited number of
theaters) while inhibiting the typically rapid sales declines for “wide-release ” (i.e.,
blockbuster) movies.
Our article fills a void in the growing literature on critics or opinion leaders by
modeling the impact of individual critics and their opinions, both by identifying
what role any individual reviewer plays in the diffusion process of a movie and by
disentangling the direct effect of product appeal from the individual effect of expert
opinion. Although we adopt the basic idea of Eliashberg and Shugan (1997), in
which critics can have an influencer and/or a predictor role, we assess the roles of
individual critics using a formal diffusion model. By using a model that is based on
the definitions of the two potential roles of critics, we are able to parse out and
simultaneously measure the predictor and influencer roles. Although we use real-
world data from the motion picture industry to provide new empirical evidence and
404 P. Boatwright et al.
insights on the role of critics and the performance of new products, the methodology
developed here can be generalized to any industry in which expert opinion matters.
The remainder of the article is organized as follows. In the next section, we
provide a review of the literature on the role of expert opinions. Next, we propose
our theoretical model. We then describe the data and present the results of our
empirical analyses. Finally, we close with a discussion of the theoretical and
managerial implications of our findings, along with suggestions for possible future
research directions.
2 Literature review
Scholars from various fields have shown growing interest in recent years in
understanding the complex role of expert opinions in many markets—films, theater,
books, music, etc. (Caves 2000; Cameron 1995). With regard to empirical evidence
on the agreement of ordinary consumers and professional critics, numerous empirical
studies have examined the relationship between critical reviews at the aggregate
level and box office performance (Basuroy et al. 2003; Litman 1983; Litman and
Kohl 1989; Sochay 1994; Litman and Ahn 1998; Wallace et al. 1993; De Silva 1998;
Jedidi et al. 1998; Prag and Casavant 1994; Ravid 1999; Holbrook 1999).
One of the fundam ental contributions in marketing regarding the role of reviewers
is by Eliashberg and Shugan (1997), who contrasted two perspectives on the role of
film critics: the influencer perspective and the predictor perspective. An influencer or
opinion leader is a person who is regarded by other people or by a group as having
expertise or knowledge on a particular subject (Assael 1984; Weiman 1991). In
films, critics are typically invited to an early screening of the film and write reviews
before the film opens to the public. Therefore, they have more information than the
public does in the early stages of the film’s run. If critics are primarily opinion
leaders, then their views will have a significant impact on early attendance figures
and box office revenues but, not necessarily on the final outcome.
Eliashberg and Shugan (1997) proposed an alternative view of critics—the
“predictor” role—in which critics’ preferences reflect those of the audiences to
which they speak. The view s of predictor critics are leading indicators of the
ultimate success of a movie and do not influence its early run in the box office. In
order to test for this role of critics relative to the influencer role, they regressed
weekly (early and late) sales of 56 movies against the total number of reviews, the
proportion of positive or negative reviews, and the number of screens. For each
movie, they had 8 weeks of box office data. They estimated eight independent
regressions, one for each week.
It is unclear that the correlation analyses of Eliashberg and Shugan (1997) and
others (Basuroy et al. 2003) accurately reflect the role of critics, for these studi es did
not account for the inherent quality of the movies. As such, their observed
correlations between ticket sales and aggregate critical acclaim might be due to a
third factor—movie quality. It is entirely possible that good movies sell well because
they are good movies, liked by critics and by the public. Because of the confounding
factor of movie quality, a positive correlation of aggregate critics with the weekly
sales of later or early weeks by itself does not reveal a role of critics.
Reviewing the reviewers: The impact of individual film critics 405
Another potential problem with the use of aggregate reviews in the prediction of
success for a new product is the potential endogeneity in the reviews themselves. For
example, a movie based on an excellent screenplay by a famous author and featuring
popular actors led by a star director is more likely to be reviewed by a larger number
of critics and to generate more positive reviews due to its higher appeal.
Consequently, one might see a strong correlation between box-office results and
the volume and average valence of the reviews, not because of the critics ’ predictive
power and influence but because of the correlation with a common factor—the
overall appeal of the movie. By focusing on the effect of individual reviews, we
attempt to at least partially avoid this potential problem, adjusting each individual
review in relation to a consensus measure of movie quality obtained across all
critics. This way, we consider the impact of each critic’s review, after discounting for
any possible “bandwagon” effect generated by the aspects intrinsic to the movie.
Second, the analysis of Eliashberg and Shugan (1997) was based upon an
assumption about the timing of critical influence, that correlation of critical opinion
with early weeks’ sales would reflect an influencer role, while correlation with later
weeks’ sales would reflect a predictor role. One difficulty in this assumption about
timing is that early and late box office sales tend to be quite correlated across
movies; better movies sell more tickets cumulative and have higher sales both early
and late in the product lifetime. A model that ignores this correlation may lack the
power to uncover the true roles of critics. Rather than comparing early and late sales,
our model estimates the extent to which sales are drive n by outside influences and
the extent to which individual critics serve as outside influences. Our approach also
allows consumers to be influenced by critics later in the lifetime of the movie, not
just early, through the dynamics of a diffusion process, which also specifically
accounts for the correlation between early sales and cumulative sales.
Third, in analyzing the impact of critics on box office sales, we control for a
number of obvious factors that influence sales. For example, movie producers invest
heavily in advertising, in the belief that advertising impacts movie sales. Given that
advertising informs the public about a movie in a manner not unlike the role of
critics, we explicitly account for advertising in our model. For similar reasons, we
also account for the movie budget, the recognition of actors and actresses, the
number of screens on which the movie opened and other important factors that could
influence sales, and we evaluate the impact of critics only after accounting for these
factors. Because of the myriad of influences on box office sales that may be
simultaneously correlated wi th critical opinion, an analysis of the role of critics
requires a carefully constructed model.
3 Modeling the role of individual critics
We assume the classic formulation of a diffusion process proposed by Bass (1969),
y
it
¼ p
i
m
i
Y
it
ðÞþq
i
Y
it
m
i
m
i
Y
it
ðÞþ"
it
ð1Þ
in which movies are indexed by i and weeks by t=1...T
i
, and we have added an error
term "
it
N 0; s
2
i
: In this notation, p
i
is the coefficient of innovation, q
i
is that of
406 P. Boatwright et al.
imitation, m
i
is the market potential, y
it
represents box office sales of movie i in week
t, and Y
it
is cumulative sales.
1
As discussed in Bass (1969), this new product diffusion model proposes that the
timing of product purchase results from one of two buying influences. The parameter
q, the coefficient of imitation, reflects the influence of previous buyers on purchase
timing. Note in particular that q reflects specifically the impact of previous buyers,
such as network effects or “word-of-mouth” and not joint decision making of groups
of buyers. A group of friends who decide which movie to watch are making a joint
decision, while q reflects the influence of those who have already seen the movie on
those who have not. The parameter p, the coefficient of innovation, reflects
influences that are not related to the number of previous buyers. In keeping with the
diffusion model literature, we take p as representing the probabili ty that a potential
buyer adopts the new product through external influences such as advertising, and q
as network effects or “word-of-mouth,” which depends on the number of current
users. A critic’s opinion might be related to the early diffusion of the product in the
same way that advertising affects adoption. If a critic is merely good at sensing the
public’s preferences or anticipating the success of a movie, his effect will show in
the market potential (m).
We employ the Bass framework because, derived from principles of new product
diffusion, its parameters have straightforward interpretations for the applied
problem; for instance, the Bass model identifies and disentangles innovator purchase
probability from market potential, an issue of key importance in our study. For many
movies, sales decline exponentially. In this case, the Bass model simplifies to a
model of exponentially decreasing sales in which the innovator purchase probability
measures the speed of decay of the exponential, or how fast public interest curtails.
One of our goals is to assess which critics’ opinions are correlated with the innovator
purchase probability (or decay of movie sales) and which are correlated with market
potential. The Bass model also explicitly measures the word-of-mouth effect, which
has been found to have a strong effect on movie adoption (Godes and Mayzlin 2004;
Sawhney and Eliashberg 1996).
We recognize that the Bass structure is not a perfect match for movies, in that the
Bass model was developed for durable goods rather than repeat-purchase products.
However, repeat purchase is rare in the movie industry and has been generally
deemed safe to ignore in the movie modeling literature (Eliashberg et al. 2000;
Sawhney and Eliashberg 1996). An alternative to the Bass model for this data would
be the generalized gamma distribution (Ainslie et al. 2005 ). For exponentially
decreasing data, a common sales pattern for movies, the Bass model and the
generalized gamma are mathematically identical, in that both simplify to the
exponential distribution for exponentially decreasing sales data. Not all movie sales
data are exponentially decreasing, so the two models are distinct across movies of
varying diffusion patterns.
To allow for variation in the role of critics across different types of products, we
specify a model with k product clusters. Within each product cluster, we assume the
Bass parameters to be a linear function of exogenous variables such as movie
1
Although Eq. 1 allows for negative sales, constraints on the space of estimated parameters (e.g. that
market potential m
i
must exceed cumulative observed sales for movie i) ensure positive sales.
Reviewing the reviewers: The impact of individual film critics 407
characteristics, critic reviews, and marketing expenditures in promoting and
distributing the movie.
φ
i
s
i
¼ W
si
θ
s
þ u
si
j
u
si
N 0; V
s
ðÞ
φ
i
¼ logit p
i
ðÞ; q
i
; log m
i
ðÞ½
ð2Þ
In Eq. 2,
i
collects the Bass parameters for movie i; we transform the Bass
parameters so that the Gaussian error assumption is reasonably accurate. W
si
is a set
of exogenous variables including movie characteristics and other relevant factors for
movie i; s
i
indexes the movie cluster and takes values 1, 2,...k. Because the
composition and dimension of the set of movie variables that affect the Bass
parameters may differ by cluster, we index W by cluster as well as by movie.
Similarly, we allow the covariance among the Bass parameters to differ by cluster.
In a study of the roles of individua l critics, the matrix of exogenous variables W
is
may be quite large as a result of a large population of individual critics. Furthermore,
individual critic’s opinions may be correlated with one another and with character-
istics of the movies, meaning that W
is
may be multicollinear. We therefore
incorporate a stochastic search variable selection (SSVS) algorithm to identify
promising subsets of variables that influence Bass parameters within a movie cluster.
The matrix W
is
contains not only the critiques of movie reviewers but also the
characteristics of movies such as its production and promotion budget. Because
empirical work with movies has already identified important movie characteristics,
we specify a framework that will incorporate such variables in the model outside the
stochastic selection process. To this end, we partition W
s
into two subsets, Z
s
and X
s
,
in which those variables in Z
s
are included with probabi lity 1, and those variables in
X
s
are possibly included in the model. Expanding Eq. 2, we have
φ
i
s ¼ Z
si
y
s
þ X
si
β
s
þ u
si
j
ð3Þ
We use a Gaussian prior on vec(y
s
); note that y
s
and β
s
are matrices of regression
coefficients. For our prior on vec(β
s
), we use the mixture framework developed by
George and McCulloch (1993):
b
sj
g
sj
1 g
sj
N 0; t
2
sj
þ g
sj
N 0; c
2
sj
t
2
sj
ð4Þ
P g
sj
¼ 1
¼ 1 P g
sj
¼ 0
¼ z
sj
ð5Þ
where β
sj
is the jth element of vec (β
s
). Here, c
sj
>1; the intuition is that each
coefficient β
sj
is either close to zero, in which case it can be safely ignored, or not
close to zero, in which case it should be part of the model . If β
sj
are close enough to
zero to be ignored, γ
sj
=0. Because the set of influential critics may vary by movie
cluster, we allow γ to vary by cluster s and critic j.
The SSVS algorithm is influenced by tuning parameters c and τ, which are set to
reflect the goals of the analysis. For instance, one goal would be model parsimony.
Our goal is to identify individual critics that may influence the diffusion process; the
most parsimonious model would leave out marginally influential critics, although we
at least want to identify critics that have a reasonably large probability of being
influential. For details on the relationship between model parsimony and tuning
parameters, see George and McCulloch (1993).
408 P. Boatwright et al.
Note that the SSVS analysis differs greatly from the standard approac h of
identifying a best single model and using that model to understand a research
problem. The SSVS recognizes uncertainty in identifying a single “true” model and
puts probability mass over a distribution of models. The traditional approach
assumes that the researcher can identify with certainty the one true model, either by
the use of theory or as an outcome of highly informative data. With the nature of our
dataset, it is unlikely that the data will clearly identify a single true model. In fact,
we know with certainty that there is not adequate information in the data to identify
a single true model for one potential situation—if a cluster has fewer observations
than it has variables, multiple different models will each fit the data perfectly. The
extreme case of fewer observations than variables illustrates an important point: that
the more variables one has for model selection, the greater the uncertainty. Because
we investigate a large set of critics, it is imperative that we allow for uncertainty of
the model.
Because SSVS models are not well known in the marketing literature, we find it
important to distinguish SSVS from a more well-known approach in data mining. In
data mining, a common analysis method is to use the search tools to identify a single
best model. This approach has been criticized because it uses a large set of variables
to identify a single model, and it is always possible to find a model that fits data well
if one has enough variables for that model. The SSVS approach produces an entirely
different outcome—a distribution over a large set of models, not a single model.
Rather than conditioning on a single chosen model, as is commonly done elsewhere,
we integrate across the whole posterior distribution of models before reporting
parameter estimates. The result is a highly conservative approach to identifying
important individual critics.
4 Data
We analyze a sample of 466 films released between December 1997 and March
2001. For each of these movies, we collected weekly revenue and screen data from
Variety magazine. Given our focus on the diffusion process, weekly data are crucial.
The data set is fairly large and is comparable to the data that ha ve been used recently
in this literature (Basuroy et al. 2003; Elberse and Eliashberg 2003).
Several factors may affect the adoption parameters p
i
and q
i
and market size m
i
for a new movie. To select covariates for our model, we build upon the extant
empirical movie modeling research. Below is a description of the variables we
include in our study:
& Budget: Several authors have shown that the budget of a film is significantly
related to its box office performance (Litman 1983; Litman and Kohl 1989;
Litman and Ahn 1998; Prag and Casavant 1994; Ravid 1999). The trade term for
the budget figure is “negative cost” or production costs. This does not include
gross participation, which is ex-post share of participants in gross revenues, nor
does it include advertising and distribution costs. The budget data were collected
from International Movie Data Base (http://www.imdb.com) and from http://
www.baseline.hollywood.com.
Reviewing the reviewers: The impact of individual film critics 409
& Advertising: Ad expenditure data for each movie in our sample were separately
collected from various issues of Leading National Advertisers (1998–2001). Our
measure of ad expenditure includes spending on all major media, including
television, radio, and print.
& Stars: Studies by Wallace et al. ( 1993); Litman and Kohl (1989), and Sochay
(1994) found that the presence of stars in the cast had a significant effect on film
rentals. On the other hand, Litman (1983) found no significant relation between
the presence of a superstar in the cast of a film and its box office rentals. Smith
and Smith (1986) found that winning an award had a negative effect in the 1960s
but a positive effect in the 1970s. Similarly, Prag and Casavant (1994) report
conflicting evidence regarding stars, with star power positively impacting a
film’s financial success in some samples but not in others. The lack of a star-
power effect has been documented in later studies as well (DeVany and Walls
1999; Litman and Ahn 1998; Ravid 1999). For star power we use the proxies
suggested by Ravid (1999). For our first definition of a “star,” we identified all
cast members who had won an Academy Award (Oscar) for Best Actor or Best
Actress and all directors who had won for Best Director in years prior to the
release of the current film. We created a dummy variable, WONAWARD, which
denotes films in which at least one actor/actress or the director has won an
Academy Award in years prior to the release of the film. To create an additional
similar measure, we defined an additional variable, NOMAWARD, which takes a
value of 1 if at least one of the actor/actress/director had been nominated for an
award prior to the release of the film.
& MPAA ratings: Ratings are considered by the industry to be an important issue.
Ravid (1999) found ratings to be significant variables in his regressions. In our
analysis, we code the ratings from http://www.mpaa.org using dummy variables.
In our sample, the proportions of G, PG, PG13, and R are 4.3, 10.0, 32.2, and
53.5%, respectively. This distribution of all films released between 1999 and
2000 (see http://www.mpaa.org) is as follows: 5% G, 8% PG, 17.6% PG13,
and 69% R. We used a dummy variable “<R” to indi cate films that had ratings of
G, PG, and PG13.
& Sequel: One variable that may be of interest is whether or not a film is a sequel
(Litman and Kohl 1989; Prag and Casavant 1994; Ravid 1999; Ravid and Basuroy
2004). Although sequels tend to be more expensive and sometimes bring in lower
revenues than the original film, they may still outperform the average film if they
can capitalize on a successful formula. The SEQUEL variable receives a value of 1
if the movie is a sequel to a previous movie and 0 otherwise.
& Number of screens: The number of screens on which a movie plays each week has
been reported to have the most significant impact on weekly revenues (Basuroy et
al. 2003; Elberse and Eliashberg 2003). We incorporate the weekly screen count
provided by Baseline.
& Movie appeal: Kamakura et a l. (2006) developed an approach that uses
information available from every expert, including those who are silent about
the product, to obtain a consensus measure of expert opinion on movie quality.
Their measure is not simply an aggregate of opinion, because the meaning of an
opinion (positive, negative, etc.) varie s by critic; for instance, the fact that an
expert is silent about a product may imply a positive or a negative review,
410 P. Boatwright et al.
depending on the expert. On e of their important findings concerned the
dimension of the latent space of judgments about the movies. Althoug h their
model allows this consensus measure to be multidimensional—where the judged
dimensions for movies, books, and plays could be entertainment value, acting, or
complexity of the plot—they found the consensus measure in movies to be
unidimensional. In other words, critics were fundamentally assessing the same
underlying latent factor when writing their opinions. We use the label “appeal” to
refer to the univariate latent factor.
& Individual reviews: The key issue in our study is to identify individual critics
whose reviews impact box office sales. On the first weekend of a film’s opening,
Va r i et y lists the reviews from four major cities: New York, Los Angeles,
Washington DC, and Chicago. Variety classifies these reviews as “pro” (positive),
“con” (negative), and “mixed.” The number of reviewers for the sample of 466
films in our data was quite large—more than 150 reviewers, each with a unique
valence. We shortened this list to 46 critics, using media circulation statistics to
select those critics whose reviews would be most widely accessible. We used all
critics whose reviews appear in publications with 2002 circulation that exceeded a
half million copies plus a few well-known syndicated critics. We used a single
statistic for each critic to measure the valence of a critical review effect of that
critic. If the review was positive, we code the critic’s effect to be 1. If negative, we
code it to be −1. Otherwise, we code it as 0 for a mixed review. An alternative
coding would be to use two variables per critic, coding the presence of a review
separately from its valence. In this coding, the presence variable would be 1 if the
critic reviewed the movie, 0 if the critic was silent about the film. The valence
would be equal to 1 if the critic was positive about the film, 0 for a mixed review,
and −1 for a negative review. We used the more crude measure so that there would
be only 46 variables representing critics, rather than double that number. Out of the
46 critics we used in our analysis, Lawrence Van Gelder (New York Times) had the
fewest number of reviews, 14, while Peter Travers (Rolling Stone) had the largest
number of reviews, 358. The most negative critic was Elvis Mitchell (New York
Times), who gave positive reviews in 10 of 63 films that he reviewed. The most
positive reviewer was Kevin Thomas (LA Times), who gave 90 positive reviews
out of a total of 123 reviews.
In light of the findings of prior empirical work regarding factors known to be
relevant to movie sales, we define the Z matrix (see Eq. 3) to be composed of the
above factors, except for individual critics, because the Z matrix includes those
factors already known to affect movie diffusion. In a later section, when we
introduce individual critics into the model, we put the individual critic variables into
the X matrix (Eq. 3). By doing so, we estimate the marginal contribution of
individual critics beyond that of the covariates identified in the extant literature. In
the next section, we describe model estimation and the results.
5 Model estimation and results
Our hierarchical model has two levels of parameters, estimated simultaneously
(please refer to the Appendix for the estimation algorithm). The first level of
Reviewing the reviewers: The impact of individual film critics 411
parameters comprises those of the Bas s model, parameters which summarize the
sales diffusion curve for the movie. The second level of parameters reveals how
exogenous variables, such as promotion budget or movie appeal, influence the Bass
model parameters. Because of the large number of movies, we do not report the
movie-specific parameters (the Bass model parameters) for any of the models we
estimate. As for fit statistics for the models of movie sales, the median R
2
across
movies is 0.98. Although these parameters provide extremely close fits to the movie
sales curves, our focus in this article is neither fit nor prediction of the movie sales
diffusion, but rather an understanding of how the diffusion parameters, which
themselves are estimates, vary across movi es.
We begin by ignoring product differences, in that we assume all movies to belong
to the same movie cluster, following the assumptions made in many previous studies.
Table 1 contains the estimates for the y parameters, measuring the impact of movie
characteristics on the diffusion of a new movie. Values of the psi (y) parameter that
are bolded are those for which the 95% posterior parameter interval does not contain
zero (i.e., in classical statistics terminology, the predictor has a “statistically
significant” impact on the diffusion process). As discussed earlier, m is the market
potential, q is the word-of-mouth parameter, and p in this context (due to the large
number of movies with exponentially decreasing sales) reflects both the opening
week’s sales (as a percentage of total box office sales) and the sales decay rate.
Table 1 Model with no movie clusters or individual critics
Z Bass parameter Ψ std(Ψ)
Intercept p −3.621 (0.124)
Intercept q 0.322 (0.022)
Intercept m −1.891 (0.116)
Log screens p 0.384 (0.023)
Log screens q −0.029 (0.004)
Log screens m 0.124 (0.025)
Log budget p 0.068 (0.04)
Log budget q 0.003 (0.005)
Log budget m 0.16 (0.041)
<R rating p −0.331 (0.061)
<R rating q −0.011 (0.008)
<R rating m 0.288 (0.069)
Advertising p −0.003 (0.047)
Advertising q −0.03 (0.007)
Advertising m 0.59 (0.049)
Appeal p −0.317 (0.034)
Appeal q 0.002 (0.004)
Appeal m 0.283 (0.039)
WonAward p −0.019 (0.065)
WonAward q −0.006 (0.008)
WonAward m 0.007 (0.074)
Sequel p −0.032 (0.12)
Sequel q −0.017 (0.015)
Sequel m 0.652 (0.14)
Values in bold indicate that the 95% posterior interval for the parameter does not contain 0. Numbers in
parenthesis are posterior standard deviations.
412 P. Boatwright et al.
The estimates reported in Table 1 lead to the following conclusions regarding the
diffusion process for all movies:
& The initial number of screens accounts for variation of all three Bass parameters.
The greater the number of opening screens, the more rapid the decline of sales,
because the effect on p is positive. In addition, movi es with a greater number of
opening screens have less of a word-of-mouth effect (negative effect on q) and a
greater total market potential (positive effect on m).
& Movies with greater advertising support are those with larger market potentials
and less of a word-of mouth effect. Whenever theater capacities limit initial sales,
advertising cannot increase early weekly box office sales (a positive effect on
Bass parameter p) but rather should slow the sales decay (a negative effect on p)
of the movie. In this model, the coefficient is estimat ed to be close to zero,
indicating no significant impact of advertising on p.
& The role of movie appeal on initial sales should be similar to advertising, either
directly stimulating early sales (see the discu ssion earlier of the coefficient of
innovation) or slowing the sales decline, depending upon the type of movie. In
this model, we find that sales fall off less rapidly (negative effect on p) for
movies with greater appeal, and that movies with greater appeal have larger
market potentials.
& Movies with greater market potential are those with larger budgets, sequels,
greater advertising support, a large number of opening screens, of greater appeal,
and those with an MPAA rating other than “R”; the latter also have sales curves
that decay less rapidly.
6 Accounting for product differences
In order to investigate product differences, we next allow for more than one cluster
of movies. The literature discusses two types of movies, “wide-release” and
“platform-release” (Ainslie et al. 2005 ; Sawhney and Eliashberg 1996). These two
movie types correspond to two different release strategies. In the wide-release
strategy, movies open in a large number of theaters, whereas platform movies open
in select theaters that are frequented by the movie’s target demographic. These two
strategies can clearly be seen in the bimodal distribution shown in Fig. 1, which
shows a histogram of the number of opening screens for the 466 movies in our data.
Seeing that the platform-release data are well below 500 opening screens and the
wide-release movies are well above 500 openin g screens, we use 500 screens to
define the two clusters: that is, a movie is defined as platform release if it opened on
less than 500 screens. This classification of movies into two categories is so clear-cut
that an attempt to cluster-analyze our data based on all movie characteristics led to
essentially the same classification as the one we report here.
Descriptive statistics of the clusters are contained in Table 2. By definition, the
clusters differ in the number of opening screens (2,272 vs 54). More generally, the
clusters differ in scale; the wide-release cluster’s 317 movies tend to have high box
office sales, larger budgets ($48.1 million vs $11.7 million), and more advertising
($16.7 million vs $3.5 million). All of the sequels are in the wide-release cluster,
Reviewing the reviewers: The impact of individual film critics 413
possibly because sequels tend to be made for previously successful films. In general,
critics are more positive when rating movies in the platform-release cluster (61%
positive ratings) than in the wide-release cluster (32% positive ratings), although
ratings exhibit high variability. These statistics suggest that critics have preferences
that do not agree with those of the viewing public (Holbrook 1999; Kamakura et al.
2006). Many of the films in the platform-release cluster cater to a smaller segment of
viewers by design; it may be that critics also tend to like such fil ms more than those
designed for the “masses” (Holbrook 1999). Our model results (see below) find
agreement between critics and the masses, in that the critical consensus is positively
correlated with the market potential.
Another important distinction between the two clusters is the shape of the time
series sales function. The wide-release cluster has a greater proportion of sales
0 1000 2000 3000
0 20 40 60 80 100 120
Number of O
p
enin
g
Screens
Frequency
Fig. 1 Histogram of number of opening screens. The spike near 0 in this chart reflects the large number of
films with fewer than 100 opening screens (see Table 2). All films in our data open in greater than 0
screens
Table 2 Descriptive movie statistics by cluster
Class Statistic Platform-release Wide-release
Number 149 317
% sequels 0% 8%
Number of weeks 14.0 (7.4) 13.6 (6.0)
Ratings G 2.7% 5.0%
PG 6.0% 12.0%
PG13 18.1% 38.8%
Awards WonAward 24.2% 34.7%
NomAward 46.3% 53.3%
Scale Budget (millions) 11.7 (14.3) 48.1 (33.1)
Advertising (millions) 3.5 (5.8) 16.7 (7.8)
Opening screens 54 (95) 2,272 (672)
Critics Number of reviews 12.8 (3.8) 14.3 (4.2)
Perc pos reviews 61% (23%) 32% (26%)
Sales Sales (millions) 8.2 (1.4) 10.6 (0.9)
First week (%) 11.4% (15.8%) 37.2% (14.9%)
Values within parentheses are standard deviations.
414 P. Boatwright et al.
(37.2% vs 11.4%) occurring in the opening week. An examination of the sales
curves revealed that 128 of the 149 movies in the platform-release cluster were
sleeper movies (those for which the mode of weekly sales occurs after the second
week) (86%), whereas only 41 of 317 (13%) of movies in the wide-release cluster
were of the sleeper shape. The difference in diffusion shape is important for the
interpretation of the coefficient of innovation p, as discussed earlier in the model
section. In the wide-release category, which is dominated by exponentially
decreasing sales diffusion curves, the coefficient of innovation p reflects this
exponential sales decay; in the platform-release category, that parameter maintains
the typical interpretation of the coefficient of innovation in the Bass model.
Although a high p is generally viewed in this industry as undesirable,
2
a high p is
desirable for sleeper movies, b ecause it triggers the diffusion of the movie through
contagion or imitation. It is for movies with exponentially declining sales that p is
undesirable, in that movies with a high p die more quickly.
In Fig. 2 we show a scatter plot of total sales versus first-week sales, in which a
triangle plotting symbol was used for wide-release movies and a circle for platform-
release movies. The cluster of platform-release movies runs along the bottom of the
plot, because its first-week sales are a relatively small percentage of total sales. The
wide-release cluster appears in the upper right; movies in this cluster have both
higher total sales (shown in log scale in the figure) and higher first-week sales.
Among other things, Fig. 2 reveals the importance of our modeling assumptions: (1)
2
Noted by one of the anonymous referees.
Lo
g
Box Office Sales
First Week Percent Sales
6 8 10 12
0.0 0.2 0.4 0.6 0.8
Fig. 2 Initial and total box office sales. Triangles are movies that opened in at least 500 screens; circles
represent movies that opened in fewer than 500 screens
Reviewing the reviewers: The impact of individual film critics 415
that coefficient of innovation p and market potential m are correlated, and (2) that
correlations of p (which will roughly correspond to first-week percent sales) and m
(which will roughly correspond to total sales) might differ by cluste r. Put more
succinctly, V
s
is non-diagonal and could be cluster dependent, and models should
account for the correlati on of p and m rather than assuming that early and late sales
are independent of each other.
After accounting for product differences, we obtain different estimates (shown in
Table 3) from the previous ones, as well as a richer description of the movie sales
diffusion process. Before accounting for product differences, movies with greater
advertising support proved to be those with larger market potentials and less of a
word-of-mouth effect, and there was no relationship between advertising and the
coefficient of innovation (p). After allowing for product differences, we found for
the wide-release cluster that advertising was negatively correlated with p. For the
wide-release cluster, the dominant model is exponentially decreasing sales, and
advertising is found to dampen the decay rate of the sales. For the platform cluster,
we did not find a strong effect of advertising on the coefficient of innovation (p), but
the more heavily advertised films ended up with less of an effect from word of
mouth. For both types of movie, those with greater advertising support had greater
market potentials.
Table 3 Model with clusters, no individual critics
Z Bass parameter Platform cluster Wide-release cluster
Ψ std(Ψ) Ψ std(Ψ)
Intercept p −3.788 0.359 −0.955 0.679
Intercept q 0.431 0.071 0.376 0.082
Intercept m −1.239 0.220 −9.529 0.812
Log screens p 0.160 0.079 0.095 0.104
Log screens q −0.048 0.018 −0.046 0.012
Log screens m 0.035 0.053 1.295 0.125
Log budget p 0.162 0.115 0.032 0.047
Log budget q 0.130 0.025 −8E-05 0.005
Log budget m −0.116 0.068 0.084 0.057
<R rating p −0.507 0.217 −0.241 0.056
<R rating q −0.094 0.050 −4E-04 0.006
<R rating m 0.384 0.169 0.182 0.069
Log advertising p 0.172 0.120 −0.126 0.055
Log advertising q −0.163 0.022 −0.001 0.006
Log advertising m 0.719 0.077 0.229 0.068
Appeal p −0.097 0.178 −0.266 0.029
Appeal q −0.014 0.035 0.001 0.003
Appeal m 0.304 0.121 0.298 0.036
WonAward p −0.192 0.232 −0.013 0.057
WonAward q −0.093 0.068 −0.002 0.006
WonAward m 0.284 0.192 0.022 0.071
Sequel p –– 0.102 0.099
Sequel q ––−0.010 0.010
Sequel m –– 0.368 0.123
Values in bold indicate that the 95% posterior interval for the parameter does not contain 0. Numbers in
parenthesis are posterior standard deviations.
416 P. Boatwright et al.
In the wide-release cluster where the dominant model is exponentially decreasing
sales, the role of movi e appeal exactly followed that of advertising; movies with
greater appeal had not only a larger market potential, but also a less pronounced
decay in sales (negative coefficient on p). In the platform cluster, greater appeal
again led to larger market potentials but did not affect the diffusion of the new
movie. We had expect ed movies with greater appeal to receive more word of mouth,
but we did not find movie appeal to be significantly related to the word-of-mouth
parameter (q) in either of the two clusters.
Although our model without product differentiation found movies with larger
production budgets to have greater market potential, the movie budget no longer had
an effect on market potential after accounting for product differences. The earlier
result was driven by differences in the type of movie; platform movies tended to
have both lower overall sales and low er budgets than wide-release movies. This
result show s the importance of accounting for different products or marketing
strategies, in that the former model attributed an effect to a movie budget that
actually turned out to be an effect of the product release strategy. We also found that
platform movies with greater budgets had a larger word-of-mouth effect, a
relationship that disappeared once we included individual critics (discussed later).
The initial number of screens was the only factor in the first model (without
product differentiation) that accounted for variation of all three diffusion parameters.
After controlling for product differences, the number of screens was no longer
associated with all three Bass parameters in either cluster. In both clusters, movies
with a greater number of opening screens had less of a word-of-mouth effect. In the
wide-release cluster, movies with a greater number of screens had greater market
potential, a result not reflected in the platform-release cluster. In contrast, movies in
the platform-release cluster with a greater numbe r of opening screens had greater
initial sales.
In the platform-release cluster, the MPAA ratings accounted for variation in all
three Bass parameters. Movies with ratings other than “R” opened with lower sales,
had lower word-of-mouth effect, but had greater overall market potential. In the
wide-release movie cluster, movies with other than “R” rating had both higher
market potential and a sales curve that did not decay as rapidly. Finally, there were
no sequels in the platform-release cluster, and the result from the previous model
(sequels have greater market potential) was confirmed in this model for the wide-
release cluster.
One overall conclusion from our examination of summary statistics is that it is
important to account for product differences when studying the diffusion of new
movies. For example, we found that advertising and movie appeal affected both sales
decay and total market potential in the wide-release cluster but affected only market
potential in the platform cluster. More generally, we found that the use of clusters
allows for accurate and meaningful parameter interpretation.
7 Product differences and individual critics
The addition of individual critics resulted in few changes in existing parameters, as
can be seen by comparing the full model (Table 4) with the previous results
Reviewing the reviewers: The impact of individual film critics 417
(Table 3). As alluded to earlier, for the platform-release cluster, movie budget in the
final model is no longer correlated with the word-of-mouth parameter. Possibly the
budget accounted for some additional aspects of movie appeal not fully covered by
our measure of appeal but reflected in the reviews of individual critics. Also in the
platform-release cluster, where seating capacity constraints are less binding, greater
advertising was shown to increase early sales. Finally, whereas the MPAA ratings
affected all three diffusion parameters in the previous model, none remained
significant in this model.
3
In the wide-release cluster, the only change in the results
was that the opening number of screens was no longer associated with the word-of-
mouth parameter.
In this final model we also investigated which critics offer unique views that
impact box office sales (i.e. which were influential after taking into account the
general consensus). Because the aggregate set of critics was used to determine the
latent appeal of a movie, these results (Table 5) for individual critics reflect their
relative (beyond the consensus) impacts rather than absolute impacts on the sales
diffusion. There may be critics that do not offer any unique insights beyond the
3
The coefficient estimates and standard errors are very close to previous values but no longer
“significant,” in that 95% posterior intervals in this final model contain 0.
Table 4 Model with clusters and individual critics
Z Bass parameter Platform cluster Wide-release cluster
Ψ std(Ψ) Ψ std(Ψ)
Intercept p −4.359 (0.411) −1.308 (0.676)
Intercept q 0.543 (0.134) 0.173 (0.193)
Intercept m −1.280 (0.216) −9.472 (0.794)
Log screens p 0.256 (0.094) 0.150 (0.104)
Log screens q −0.070 (0.028) −0.014 (0.029)
Log screens m 0.017 (0.055) 1.281 (0.122)
Log budget p 0.203 (0.114) 0.016 (0.045)
Log budget q 0.026 (0.031) −0.010 (0.011)
Log budget m −0.003 (0.062) 0.095 (0.055)
<R rating p −0.488 (0.26) −0.225 (0.056)
<R rating q −0.131 (0.08) 0.010 (0.015)
<R rating m 0.393 (0.17) 0.165 (0.068)
Log advertising p 0.204 (0.109) −0.131 (0.057)
Log advertising q −0.072 (0.036) −0.002 (0.014)
Log advertising m 0.666 (0.075) 0.238 (0.069)
Appeal p −0.129 (0.178) −0.253 (0.03)
Appeal q 0.003 (0.056) 0.008 (0.008)
Appeal m 0.336 (0.12) 0.291 (0.036)
WonAward
a
p –– – –
WonAward q –– – –
WonAward m –– – –
Sequel p –– 0.088 (0.098)
Sequel q –– −0.031 (0.025)
Sequel m –– 0.372 (0.122)
Results for individual critics appear in the following table. Values in bold indicate that the 95% posterior
interval for the parameter does not contain 0. Numbers in parenthesis are posterior standard deviations.
a
Because WonAward is consistently insignificant, it was dropped from this model.
418 P. Boatwright et al.
readily apparent movie quality and critic consensus; these critics without unique
information will not appear to be influential or predictive in our analysis, because
their impact is already embedded in the consensus.
Unlike the elements of the y vector, which correspond to movie characteristics
determined by previous research to influence movie sales, the prior for elements of
the β vector is a mixture model (Eq. 4 )—a mixture of two distributions—which
reflects two cases: (1) that the impact of critic j on movies in cluster s is negligible,
in which case γ
sj
=0; and (2) that the impact of critic j on movies in cluster s is
substantial, in which case γ
sj
=1.
Empirically, γ
sj
will be between 0 and 1; this is the probability that critic j
influences movies in cluster s. For any single model, γ
sj
is either 0 or 1. The average
across models is a marginal probability that reflects a critic’s influence across the
distribution of possible models. Table 5 lists all cases for which γ
sj
exceeds 0.5. If γ
sj
>
0.5, the evidence shows that that critic j most likely influenced the diffusion process in
cluster s; otherwise, the evidence reveals that the critic most likely did not influence
the diffusion process. Also note that each γ
sj
corresponds to a particular β
sj
,andβ
sj
influences one or more specific Bass parameters. As implied in the format of Table 5,
which reports both γ
sj
and β
sj
,theγ
sj
values reveal the nature of each critic’s
relationship to the diffusion process—that is, whether the critic’s opinion correlates
with the coefficient of innovation p, the coefficient of imitation q, or overall market
potential m.
In the platform-release cluster the dominant sales pattern is the ‘sleeper’ diffusion
pattern, and higher p’s engender earlier sales, there by triggering the diffusion process
via imitation. John Petrakis is among those that appear to be influential (
g >0.50).
The positive estimate of β shows that Petrakis’ reviews correlate positively with the
innovation coefficient p, suggesting that a positive opinion about a movie leads to an
increase in early ticket sales for the movie. Michael Wilmington, Manohla Dargis,
and FX Feeney are even more close ly related to the diffusion process, because their
g estimates are higher than that of Petrakis. As in the case of Petrakis, their reviews
are positively correlated with the coefficient of innovation. The data quite strongly
show the relevance of Owen Gleiberman’s opinion, in that
g for Gleiberman is
estimated to be 0.909. Gleiberman’s opinions run counter to the tastes of the viewing
public—it is when his reviews are negative that early ticket sales increase.
As discussed earlier, the role of the innovation coefficient p in the wide-release
movie cluster is distinct from the other cluster. Due to the exponentially declining
sales pattern, typical of “blockbuster” movies, advertising and movie appeal were
Table 5 SSVS results for individual critics
Cluster Critic Bass param.
g
sj
β std(β)
Platform Petrakis p 0.534 1.045 (0.495)
Platform Wilmington p 0.563 0.466 (0.182)
Platform Dargis p 0.753 0.552 (0.199)
Platform Feeney p 0.754 0.726 (0.271)
Platform Gleiberman p 0.909 −0.620 (0.193)
Wide-release Howe p 0.597 −0.107 (0.032)
Wide-release Van Gelder p 0.756 0.217 (0.067)
Numbers in parentheses are posterior standard deviations.
Reviewing the reviewers: The impact of individual film critics 419
found to slow the sales decay of a movie (to correlate negatively with p) rather than
to bring higher early sales. In this cluster, Desson Howe and Lawrence Van Gelder
were found to be influencers. A positive opinion from Howe would slow the decline
of early sales (a negative β coefficient). A positive opinion from Van Gelder
hastened the sales decline, indicating that Van Gelder’s preferences, like Gleiberman’s,
run counter to the market.
It is important to recognize that the views of every relevant individual critic were
found to be associated with the coefficient of innovation rather than with market
potential. Fo r market potential m, none of the critical reviews’
g
sj
exceeded 0.5; the
largest marginal posterior mean was a 0.20.
4
The evidence indicates that the
marginal information from individual critics is not correlated with market potential.
In the terminology of Eliashb erg and Shugan (1997), our results show that a few
critics are influencers; none in our data are predictors.
In the theoretical framework of Eliashberg and Shugan (1997), critics that are
predictors may have opinions that correlate with the tastes of the readers of their
critiques. Our results indicate that c ritics may be influential even while having
opinions that are opposite of those of their audience. In writing their reviews, critics
provide information about movies that allows their consumers to form their own
quality expectations apart from the overall opinion of the critic, allowing critics to be
influential in a market whose tastes do not correlate with their own.
In addition, we found more influential critics for platform movies than for wide-
distribution movies. Although the discrepancy between five influential critics for the
platform cluster and two for the wide release cluster is relatively small, one would
expect critics to have greater informational value for the less advertised platform-
release movies. Possibly the larger number of influential critics for the platform
cluster reflects the need for information for that type of movie.
8 Discussion and managerial implications
Marketing research on critics’ impacts on box office performance has so far
examined the aggregate effect of critics, which is problematic because the aggregate
critic effect is confounded with the underlying appeal of the movie. In order to assess
the role of critics as well as account for the underlying appeal of the movie, this
research investigated the impacts of individual film critic’s reviews on box office
performance of motion pictures after controlling for the appeal by using the latent
appeal measure of Kam akura et al. (2006). This correction for the overall appeal of a
movie is important, because some individual critics might be more likely to write
positive reviews of movies with broader appeal, producing a spurious correlation
between their opinions and ticket sales. Bec ause the latent appeal measure of
Kamakura et al. (2006) is the consensus of the set of critics, our estimates for
individual critics reveal how unique contributions of individual critics marginally
impact box office sales. We find that certain critics in their role as opinion leaders are
correlated with the coefficient of innovation in the Bass model framework, a view
4
For q, the largest was 0.12.
420 P. Boatwright et al.
analogous to the influencer perspective (Eliashberg and Shugan 1997). Overall, we
find critics to be influencers, and not predictors— conclusions that are opposite of
those of Eliashberg and Shugan (1997) and closer to those of Basuroy et al. (2003).
Rather than separately investigating how early box office sales and total box
office sales are influenced by covariates as in previous studies, our model accounts
for the information sources in the marketplace about new products, using the well-
known Bass model for the diffusion of innovations. Whereas the marketing literature
already contains extensive research applyi ng diffusion theory to the study of the
spread of new ideas and products in a wide variety of settings, research exploring the
impact of information from opinion leaders on the diffusion process has been
limited. Recently, however, researchers in public policy and health have considered
different methods of accelerating the diffusion of innovation s using opinion leaders
(Valente and Davis 1999).
Another distinguishing aspect of our approach that sets it apart from previous
work is that incorporating the covariates—movie appeal and individual critic’s
impact—allows us to distinguish between the two types of movies while at the same
time accounting for the possibility that both ticket sales and critics’ opinions might
be correlated with movie quality. The Bass diffusion model has three parameters and
thus can speak to more than simply early and late sales. In particular, the diffusion
model has a parameter that can be interpreted as a word-of-mouth effect (q). Most of
our results show impacts on either initial sales or market potential, confirming the
validity of the framework proposed by Eliashberg and Shugan (1997), in which
initial sales and late sales (total sales) are the dominant features.
After accounting for the relationship of early sales and tota l sales, and after
recognizing that this relationship varies across platform-release and wide-release
movies, we find that the roles of covariates differ for these two types of movie. Note
that because our model accounts for the interrelationship of m and p, we are able to
estimate effects on both m and p in spite of their high correlation with each another.
Our results show that advertising hastens adoption (p) and increases the market
potential (m) for platform-release movies, whereas for wide-release movies,
advertising delays the decay process (p) and enhances the market potential (m).
Along similar lines, we find more critics to be influential in platform-release movies
than in wide-release movies, a result that fits with the greater availability of
information about wide-release movies, which translates into less potential reliance
on information from critics. There is some anecdotal evidence in the mainstream
media that corroborates these findings. Writing about wide-release movies, movie
critic Richard Corliss wrote “ Hollywood’s marketers have become tremendously
efficient at getting their core audiences to see their big movies. They don’t need
critics for that.” (Corliss 2007). As for platform release movies, Joe Morgenstern,
film critic of the Wall Street Journal, has noted “...movie critics still play a crucial
role in supporting independent films. We [critics] may well be more useful than ever
in that regard, because the independent film movement is struggling more
desperately than ever to find production money, publicity budgets, and access to
multiplex screens” (Morgenstern 2006).
Earlier results with aggregated effects of critics were unable to distinguish the
differential impacts of covariates between different types of movies. For wide-
release movies, the diffusion curve of the vast majority of films exponentially
Reviewing the reviewers: The impact of individual film critics 421
decreased, and the word-of-mouth effect was trivial. Even so, movie appeal had a
statistically significant, albeit small, effect on word of mouth in the wide-release
cluster, for the results show that positive acclaim increased word of mouth. Word of
mouth is non-trivial for the platform-release movies, for which positive acclaim
lessened the word-of-mouth effect. These results reveal that high positive acclaim
has an effect similar to advertising across movie clusters.
Although we relied upon extant research to identify different movie types, future
research may simultaneously estimate movie clusters and identify influential
individual critics. Simultaneously identifying data clusters and selecting model
variables poses a modeling and data challenge. Clustering algorithms generally rely
on some stable criterion (such as a likelihood) and identify data structures that fulfill
the criterion. Variable selection models rely on fixed datasets and identify models
(again, a likelihood) that best fit the data. Therefore, to cluster and select variables
simultaneously presents a research challenge, in that clustering and variable selection
rely on opposing assumptions. Additionally, our model conditions on certain
variables such as the number of opening screens that might be endogenously
determined, so the coefficients associated with these variables are biased away from
what their values would be in a full specification. To the extent that other analyses
may require more accurate estimates for these estimates, future research can augment
the modeling framework accordingly.
Given the nature of our estimates, where influential critics are those that offer
distinct and relev ant information beyond the readily apparent movie quality, it is not
clear a priori which critics will be found to be influencers. It is quite possible that
widely circulated reviews offer no information beyond the readily apparent movie
quality (and consensus viewpoint). Such an outcome could be consistent with
strategy literature studies on first mover s and followers that find that those with
dominant market shares have incentive to imitate rather than innovate, even for
information goods (Hidding and Williams 2002; Lieberman and Montgomery 1988).
It is also possible that it is the unique and compelling insights of the well-known
critics that have led to their wide mark et circulation. We do not specifically examine
leader–follower strategies of well known critics (e.g. Roger Ebert) who did not
appear in Table 5. For such critics, our results even so are not in support of the latter
hypothesis, in that we do not find evidence that such critics had unique insights that
influenced box office sales. Our results do identify specific critics that appear
incrementally influential, suggesting the best targets to be coddled by producers.
Further research may reveal that specific critics have demographic-specific influence
(appealing to youth, for instance), because our research did not consider that certain
reviewers may be more influential than others for specific movie genres.
5
Finally, while our analyses focused on the impact of movie critics on the diffusion
of new movies, we must re-iterate that our framework is not limited to the movie
industry; it is applicable to the broad category of “experience” goods such as music,
5
In some of our preliminary modeling, we examined differences across genres of films, not finding
differences between genres. One plausible explanation is sample size, that even with our relatively large
set of films (466), the number of films in the 17 genres (action, adventure, animation, comedy, crime,
documentary, drama, family, fantasy, foreign, horror, musical, mystery, romance, sci-fi, thriller, and
western) in the 2 clusters is on average 13. We also combined the 17 genres into 9 more general genres,
still not finding differences between them.
422 P. Boatwright et al.
restaurants, wine s, video games, books, etc. where consumers seek the opini on of
experts and vendors use critical acclaim as a promotion tool.
Acknowledgment Suman Basuroy thanks the participants of the Carl De Santis Center Conference held
at Florida Atlantic University in November 2006 for numerous comments and suggestions.
Appendix
In this appendix we specify the prior distributio ns of the model parameters. Because
we estimate model parameters with the Gibbs sampler, we also describe the
conditional posterior distributions. Convergence of the MCMC chains was assessed
with BOA version 1.0.1 (see Best et al. 1995).
For our prior on V
s
, we use the covariance prior of Barnard et al. (2000),
decomposing V
s
into a vector of standard errors (ξ
s
) and a correlation matrix R
s
.We
use an independent inverse gamma prior on each element of the standard error
vector, x
2
sl
IG n
xs
; k
xs
; and we use a unifo rm prior over the space of positive
definite correlation matrices for the prior on R
s
. To complete the specification of our
cluster-specific priors, we assume s
2
IG n
s
; k
s
ðÞand y
s
N m
ys
; w
ys
:
We estimat e model parameters using Gibbs sampling, drawing from the full set of
conditional distributions. The conjugate priors on σ
2
, y
s
, and β
s
lead to well-known
conditional distributions (inverse gamma and normal) for the following conditional
posteriors:
σ
2
i
φ
i
; ν
σ
; κ
σ
j
;
y
s
φ
i
fg
s
; s; β
s
; V
s
; μ
s
; w
s
; γ
s
; and
β
s
φ
i
fg
s
; s; y
s
; V
s
; c
s
; τ
s
; γ
s
:
To simplify the structure of the algorithm while allowing the dimension of W
s
to
vary across clusters, we simply condition on certain elements of y
s
and/or β
s
as
equal to zero; the conditional distributions of the remaining elements are still
multivariate normal, and draws are easily obtained.
Our prior for γ
sj
is
f gðÞ¼
YY
z
g
sj
sj
1 z
sj
1g
sj
ðÞ
Although this prior implies independence, this assumption has been found to
work well in practice (George and McCulloch 1993). The discrete conditional
posterior distribution γ
sj
β
s
; φ
i
fg
s
; s; y
s
; V
s
; c
s
; τ
s
; ζ
s
hi
is Bernoulli where
p γ
sj
¼ 1
¼
β
s
φ
i
fg
s
; s; y
s
; V
s
; c
s
; τ
s
γ
sj
¼ 1
hi
ζ
s
β
s
φ
i
fg
s
; s; y
s
; V
s
; c
s
; τ
s
γ
sj
¼ 0
hi
1 ζ
s
ðÞþβ
s
φ
i
fg
s
; s; y
s
; V
s
; c
s
; τ
s
γ
sj
¼ 1
hi
ζ
s
The conditional distribution of the vector φ
i
σ
2
i
; η
i
; θ
s
; V
s
is not conjugate, and
we use a Metropolis Hastings algorithm to obtain draws of each φ
i
. To make the
draws, we transformed the thir d parameter from log(m
i
) to log m
i
Y
iT
i
y
iT
i
ðÞ;
because we know that sales potential must exceed observed sales. Not only did this
Reviewing the reviewers: The impact of individual film critics 423
transformation allow us to incorporate relevant information in the draws of m
i
, but it
also ensured that predicted sales from the Bass model would be non-negative. We
used the multivariate t distribution as our proposal density, with a location set equal
to the previous draw of φ
i
and a covariance matrix equal to a
φ
V
s
; where a
φ
is a
tuning constant.
Finally, we used the Griddy Gibbs algorithm (Ritter and Tanner 1992) to simulate
from the conditional distribution of V
s
V
s
φ; η; θ
s
; ν
ξs
; κ
ξs
; s
as was done by Barnard et al. (2000).
References
Ainslie, A., Drèze, X., & Zufryden, F. (2005). Modeling movie lifecycles and market share. Marketing
Science, 24(3), 508–517.
Assael, H. (1984). Consumer behavior and marketing action (2nd ed.). Boston: Kent Publishing
Company.
Barnard, J., McCulloch, R., & Meng, X. (2000). Modeling covariance matrices in terms of standard
deviations and correlations, with application to shrinkage. Statistica Sinica, 10(4), 1281–1311.
Bass, F. (1969). A new product growth model for consumer durables. Management Science, 15, 215–227
(January).
Basuroy, S., Chatterjee, S., & Ravid, S. A. (2003). How critical are critical reviews? The box office effects
of film critics, star power and budgets. Journal of Marketing, 67, 103–117.
Best, N., Cowles, M. K., & Vines, K. (1995). CODA: Convergence diagnosis and output analysis software
for Gibbs sampling output. MRC Biostatistics Unit, Institute of Public Health, Cambridge University.
Cameron, S. (1995). On the role of critics in the culture industry. Journal of Cultural Economics, 19,
321–331.
Caves, R. E. (2000). Creative industries. Cambridge, MA: Harvard University Press.
Corliss, R. (2007) Don’t read this column! Time, April 19.
De Silva, I. (1998). Consumer selection of motion pictures. In B. R. Litman (Ed.), The motion picture
mega-industry (pp. 144–171). Needham Heights, MA: Allyn Bacon.
DeVany, A., & Walls, W. D. (1999). Uncertainty in the movies: Can star power reduce the terror of the box
office? Journal of Cultural Economics, 23, 285–318 (November).
Elberse, A., & Eliashberg, J. (2003). Demand and supply dynamics for sequentially released products in
international markets: The case of motion pictures. Marketing Science, 22(3), 329–354 (Summer).
Eliashberg, J., & Shugan, S. M. (1997). Film critics: Influencers or predictors? Journal of Marketing, 61,
68–78 (April).
Eliashberg, J., Jonker, J., Sawhney, M. S., & Wierenga, B. (2000). MOVIEMOD: An implementable
decision support system for pre-release market evaluation of motion pictures. Marketing Science, 19
(3), 226–243.
George, E. I., & McCulloch, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American
Statistical Association, 88(423), 881–889.
Godes, D., & Mayzlin, D. (2004). Using online conversations to study word of mouth communication.
Marketing Science, 23(4), 545–560.
Greco, A. N. (1997). The book publishing industry. Needham Heights, MA: Allyn Bacon.
Hidding, G. J., & Williams, J. R. (2002). Historical analysis of first-mover advantage in I.T.-driven
Markets. Tepper Business School working paper.
Holbrook, M. B. (1999). Popular appeal versus expert judgments of motion pictures. Journal of Consumer
Research, 26
, 144–155 (September).
Jedidi, K., Krider, R. E., & Weinberg, C. B. (1998). Clustering at the movies. Marketing Letters, 9(4),
393–405.
Kamakura, W. A., Basuroy, S., & Boatwright, P. (2006). Is silence golden? An inquiry into the meaning of
silence in professional product evaluations. Quantitative Marketing and Economics, 4(2), 119–141.
424 P. Boatwright et al.
Lieberman, M. B., & Montgomery, D. B. (1988). First-mover advantages. Strategic Management Journal,
9(3), 41–58.
Litman, B. R. (1983). Predicting the success of theatrical movies: An empirical study. Journal of Popular
Culture, 17, 159–175 (Spring).
Litman, B. R., & Ahn, H. (1998). Predicting financial success of motion pictures. In B. R. Litman (Ed.),
The motion picture mega-industry (pp. 172–197). Needham Heights, MA: Allyn Bacon.
Litman, B. R., & Kohl, L. S. (1989). Predicting financial success of motion pictures: The ’80s experience.
Journal of Media Economics, 2,35–50.
Morgenstern, J. (2006). Rumors of critics’ demise are greatly exaggerated. Wall Street Journal,p.6
April 29.
Prag, J., & Casavant, J. (1994). An empirical study of the determinants of revenues and marketing
expenditures in the motion picture industry. Journal of Cultural Economics, 18, 217–235.
Ravid, S. A. (1999). Information, blockbusters, and stars: A study of the film industry. Journal of
Business, 72(4), 463–492 (October).
Ravid, S. A., & Basuroy, S. (2004). Managerial objectives, the R-rating puzzle, and the production of
violent films. Journal of Business, 77, S155–S192.
Reddy, S. K., Swaminathan, V., & Motley, C. M. (1998). Exploring the determinants of broadway show
success. Journal of Marketing Research, 35, 370–383 (August).
Regan, K. (2001). Are tech analysts too powerful? E-Comm erce Times, April 10, http://www.
ecommercetimes.com/perl/story/8841.html.
Ritter, C., & Tanner, M. A. (1992). Facilitating the Gibbs sampler: The Gibbs stopper and the Griddy-
Gibbs sampler. Journal of the American Statistical Association, 87(419), 861–868 (September).
Sawhney, M. S., & Eliashberg, J. (1996). A parsimonious model for forecasting gross box office revenues
of motion pictures. Marketing Science, 15(2), 321–340.
Smith, S. P., & Smith, V. K. (1986). Successful movies—a preliminary empirical analysis. Applied
Economics, 18, 501–507 (May).
Sochay, S. (1994). Predicting the performance of motion pictures. Journal of Media Economics, 7(4),
1–20.
Valente, T. W., & Davis, R. L. (1999). Accelerating the diffusion of innovations using opinion leaders.
Annals of the American Academy of Political and Social Sciences, 566,55–67 (November).
Wallace, W. T., Seigerman, A., & Holbrook, M. B. (1993). The role of actors and actresses in the success
of films. Journal of Cultural Economics, 17,1–27 (June).
Weiman, G. (1991). The influentials: Back to the concept of opinion leaders. Public Opinion Quarterly,
55, 267–279 (Summer).
Reviewing the reviewers: The impact of individual film critics 425