Content uploaded by J. Scott Armstrong
Author content
All content in this area was uploaded by J. Scott Armstrong on Aug 07, 2018
Content may be subject to copyright.
Published in Organizational Behavior and Human Performance, 14 (1975), 257-263
The Use of the Decomposition Principle in Making Judgments
J. Scott Armstrong
The Wharton School, University of Pennsylvania
William B. Denniston, Jr.
2nd Lieutenant, U.S. Army
Matt M. Gordon
H.J. Heinz Inc., Pittsburgh, PA
ABSTRACT
One hundred and fifty-one subjects were randomly divided into two groups of roughly equal size.
One group was asked to respond to a decomposed version of a problem and the other group was
presented with the direct form of the problem. The results provided support for the hypotheses that
people can make better judgments when they use the principle of decomposition; and that
decomposition is especially valuable for those problems where the subject knows little. The results
suggest that accuracy may be improved if the subject provides the data and the computer analyzes
it, than if both steps were done implicitly by the subjects.
One of the basic notions behind the scientific method is that a problem should be explicitly "decomposed" or
broken down into a series of subproblems. Solutions are then obtained for each subproblem, and these solutions are
combined as described by Raiffa (1968, p. 271):
The spirit of decision analysis is to divide and conquer: Decompose a complex problem into
simpler problems, get one's thinking straight in these simpler problems, paste these analyses
together with a logical glue . . .
Polya (1948) provided a general discussion on the use of decomposition in problem solving and Hertz (1964)
provides an example of decomposition in the capital investment problem.
With the decomposition principle, one can generally take account of more factors than when making direct or
global judgments. In addition, it would seem possible to analyze information on the various components more
effectively by "computer" than by doing it in one's head. It is also expected that decomposition will improve
accuracy since the errors from the parts should tend to compensate for one another. In view of these advantages, the
following hypotheses are suggested on the use of decomposition relative to the use of direct estimation:
H1: Use of the decomposition principle will lead to more accurate judgments in most situations;
H2: The value of the decomposition principle is greatest where uncertainty is high.
We were unable to find much empirical evidence which was relevant to the above hypotheses. One exception was
provided by Einhorn (1972), who found that decomposition led to better predictions of life span for people with
Hodgkin's disease than could be obtained by global judgments. Below we describe a small study which was
specifically designed to test the two hypotheses.
2
METHODS
Design
In order to compare the direct and decomposed methods, it seemed worthwhile to choose problems in which there
was a "known" answer. It was also important to contrast problems about which the subjects were likely to have good
information with those where they knew little. Five problems were selected; they are presented here in order from
those which were thought to be easy to answer directly, to those which would be difficult (the answers are also
provided here).
1. How many families were living in the United States in 1970? (Answer: 57 million. Source: World
Almanac)
2. How many students do you think dropped out of high school (grades 9-12) in the United States in 1969?
(Answer: 2,690,000. Source: World Almanac)
3. How many packs (rolls) of Polaroid color films do you think were used in the United States in 1970?
(Answer: 51 million. Source: Annual Reports, Polaroid Corporation)
4. How many pounds of tobacco were produced in the United States in 1969? This includes tobacco which
was used for cigarettes, pipes, cigars, and chewing. It also includes tobacco which was exp orted.
(Answer: 1.8 billion. Source: Statistical Abstract of the United States, 1972
5. A Philadelphia radio station and a chewing gum company cosponsored a contest in 1972. The contest
asked all metropolitan Philadelphia high schools to submit as many postcards as they could with the
message "Carefree Sugarless Gum" handwritten on them. There was no limit to number of cards that
could be submitted by a single student or a single school. The school that submitted the most cards was
the location of a free rock concert by the "Grass Roots" band. The contest was heavily promoted. How
many postcards were submitted in the contest? (Answer: 66.5 million. Source: WIBG Radio Station,
Philadelphia, PA).
This ranking of difficulty was verified by asking a convenience sample of six students to rank them according to
how well they thought they could estimate the answers. Their rankings were identical with ours. Mean rankings
were 1.0 for Families, 2.4 for Dropouts, 2.8 for Film, 4.0 for Tobacco, and 5.0 for Contest. It may be noted that the
rankings were very clear-cut except for the Dropouts and Film problems.
The decomposed form of the problems was generated by two of the authors and another professor. None knew the
"true answers" when this step was carried out. The basic strategy was (1) to decompose each problem into as many
parts as were relevant to the answer and could be managed in a reasonable amount of time, (2) to make the parts of
roughly equal importance to the answer, (3) to select parts about which there was relatively good general
information, and (4) to be sure that the final answer could be obtained from the parts by simple, arithmetic
operations. As may be noted, the decomposition also relied heavily upon bounded estimates from 0 to 100%. The
decomposed questions were as follows:
1. a. What was the population of the U.S. in 1970? _____
b. How many people were there in the average family living in the U.S. in 1970? _____
2. a. What do you think was the population of the U.S. in 1969? _____
b. What PERCENTAGE of these people do you think were going to school in 1969 (all grades: kindergarten
through grad school)? _____
c. Of those people going to school in 1969, what PERCENTAGE do you think were in high school (grades
9-12)? _____
d. Of those students in high school in 1969 (of high school age), what PERCENTAGE do you think dropped
out of school in 1969? _____
3
3. a. How many people do you think were living in the U.S. in 1970? _____
b. In 1970 what do you think was the size of the average family living in the U.S.? _____
c. In 1970 what PERCENTAGE of the families do you think owned cameras? _____
e. Of those families that owned cameras, what PERCENTAGE do you think owned Polaroid cameras?
_____
f. What do you think was the average number of packs (rolls) of Polaroid film used per Polaroid camera
owner in 1970? _____
g. What PERCENTAGE of the Polaroid film used in 1970 do you think was color film? _____
4. a. What do you think was the population of the U.S. in 1969? _____
b. What PERCENTAGE of these people do you think smoked:
cigarettes _____
pipes _____
cigars _____
chew tobacco _____
c. Of those people who smoked cigarettes, how many cigarettes on the average (per person per day) do you
think they smoked? _____
d. Of those people who smoked pipes, how many pouches of pipe tobacco on the average (per person per
day) do you think they smoked? _____
e. Of those people who smoked cigars, how many cigars on the average (per person per day) do you think
they smoked? _____
f. Of those people who chewed tobacco, how many bags of chewing tobacco on the average (per person
per day) do you think they chewed? _____
g. On the average, how many cigarettes do you think there are in a pound of tobacco? _____
h. On the average, how many pouches of pipe tobacco do you think there are in a pound of tobacco? _____
i. On the average, how many cigars do you think there are in a pound of tobacco? _____
j. On the average, how many bags of chewing tobacco do you think there are in a pound of tobacco?
_____
k. What PERCENTAGE of the tobacco produced in the U.S. in 1969 do you think was exported? _____
5. A Philadelphia radio station and a chewing gum company recently co-sponsored a contest. The contest asked
all metropolitan Philadelphia high schools to submit as many postcards as they could with the message
"Carefree Sugarless Gum" handwritten on them. There was no limit to the number of cards that could be
submitted by a single student or a single school. The school that submitted the most cards was the location of a
free rock concert by the "Grass Roots" band. The contest was heavily promoted.
a. What is the population of metropolitan Philadelphia? _____
b. Of the people in metropolitan Philadelphia, what PERCENTAGE are in high school (grades 9-12)?
_____
c. Of the high school students in metropolitan Philadelphia, what PERCENTAGE participated in the
contest? _____
d. What was the average number of handwritten postcards submitted by a participant in the contest? _____
It should be noted that subjects answering the direct version of the questionnaire were free to decompose the
problems on their own. However, few subjects wrote any calculations on their questionnaires.
Subjects
Three groups of graduate and undergraduate students from classes at the Wharton School of the University of
Pennsylvania were tested. Group I was composed of 50 students drawn from three separate classes; Group II was
composed of 50 students drawn from two classes; and Group III was composed of 51 students from two classes. In
each class an attempt was made to randomly split the students into two equal subsamples; one subsample was given
the direct questions while the other was given the decomposed questions.
4
Procedure
Each of the authors and one another person was used in the adminis tration of the questionnaires. The
administrators were instructed not to discuss the purpose of the experiment to the subjects prior to, or during the
testing. The instructions and other aspects of the situation varied according to the group.
Group I was given three problems (Dropouts, Film, and Tobacco). The following instructions were read:
We're doing a study on decision making and would like you to help us out. It will take you
less than 15 min. We will be happy to let you know how close your answers are to the true
answers. Please be certain to answer every question. If you are uncertain about any answer,
give your best estimate. If you have any questions, raise your hand. Thank you for your
cooperation.
Group II was given all five problems. In addition, rewards of $15, $10, and $5 were offered for the 3 students (out
of 50 total) who provided the most accurate answers. The following instructions were read:
We're doing a study on decision making and would like you to help us out. It will take you
less than 15 min. There will be cash prizes of $15, $10, and $5 for the three students who
give the best answers. Please be certain to answer every question. If you are uncertain
about any question, give your best estimate. If you have any questions, raise your hand.
Thank you for your cooperation.
Group III was also given all five problems. The procedures for this group were the same as those for group 11
except that minor revisions were made in the wording of the questions to improve clarity, the order of the questions
was reversed, there was no reward, and the following instructions were read:
We are doing a study on decision making. While we cannot explain the purpose of this
experiment at the present time, it is very important that you try to be as accurate as
possible and that you answer every question. You will have 15 min. Please use the
complete 15 min but make sure that you have completed all of the questions in this time.
You will be notified when 10 min remain, then 5 min, then 1 min. Begin immediately.
In summary, it may be noted that certain key factors were varied over the course of the experiment. These included
the setting of the experiment (it was administered at eight different times to eight different classes by four different
administrators); the instructions; the wording of the questions; the ordering of the questions; and the use of a reward
vs. no reward.
Data Analysis
The accuracy ratio was used as the criterion. This was obtained by dividing the "estimate" by the "actual" in cases
where the estimate exceeded actual. If actual exceeded the estimate, then actual was divided by estimate. In simpler
terms, the larger of the two numbers was placed in the numerator and the smaller in the denominator. For example,
if the actual value for a given problem were 10, then answers of both 2 and 50 would have accuracy ratios of 5. The
primary advantage of this ratio was that it eliminated problems with scaling so that the errors in the different
problems might be compared with one another. Another advantage was that the loss function was symmetrical-in
other words, there was no advantage for the subject to try to bias his answers on either the high or low side.
RESULTS
The results from Groups I, II, and III are presented in Table 1. The subjects providing the decomposed estimates
did worse than those providing direct estimates on only one of the 13 comparisons. Furthermore, on 9 of the 13
comparisons the decomposed version was significantly more accurate (significant at .05 level using the median test
from Siegel, 1956). The estimates obtained using decomposition were also less likely to have large errors. For
5
example, for all groups combined, 64% of the direct estimates for the Tobacco problem were off by a factor of ten or
more, while only 20% of the decomposed estimates for this problem were off by such a degree. These results
support hypothesis H1 that use of the decomposition principle leads to more accurate estimates. There was' no
evidence that accuracy was affected by a reward. Follow-up dis cussions with each group also led to mixed reactions
as to whether the reward had any effect and, if so, what would be the direction of the effect. In any event,
decomposition led to significant improvements with or without the use of a reward.
TABLE 1
Median Accuracy Ratio: Decomposed vs. Direct Questions
Question
(1)
Original rank
(2)
Direct version
(n = 25)
(3)
Decomposed version
(n = 25)
Difference
Group 1
Dropouts
Film
Tobacco
2
3
4
5.3
21.1
103.2
2.2
2.1
2.1
3.1
190
101.1
Group 2
Families
Dropouts
Film
Tobacco
Contest
1
2
3
4
5
1.3
5.4
10.2
18.0
831.3
1.3
2.3
7.0
7.4
147.8
0.0
3.1
3.2
10.6
683.5
Group 3
Families
Dropouts
Film
Tobacco
Contest
1
2
3
4
5
1.4
5.9
5.1
9.0
351.0
1.2
6.0
2.7
5.7
184.7
0.2
0.1
2.4
3.3
166.3
The rankings of differences for Groups I and II were in perfect agreement with the hypothesis (see columns 1 and
4 in the table). While the number of questions for Group I was too small to allow for a test of statistical significance,
the rankings in Groups II and III were each significant at the .05 level (using Spearman Rank Correlation from
Siegel, 1956). Thus, the results also support H2--i.e., that the value of decomposition is greatest where the subject's
knowledge is poorest.
FURTHER RESEARCH
Further research would be useful in trying to better define how the decomposition should be carried out. For
example, does it help to use bounded questions? To what degree should the problem be decomposed? What role
does the scaling (units) of the answers have upon the results? Is it better to have the subject decompose the problem
in order to make better use of his own information? We did some exploratory work on this last question by
comparing 11 subjects who decomposed two problems with 11 subjects who were provided with a decomposed ver-
sion. The self-decomposition was poorer on each question, but the differences were not significant.
REFERENCES
Raiffa, H. (1968), Decision Analysis. Reading, Mass.: Addison-Wesley.
Polya, G. (1948), How To Solve It. Princeton, N.J.: Princeton University Press.
6
Einhorn, H. J. (1972), “Expert measurement and mechanical combination,” Organizational Behavior and Human
Performance, 7, 86-106.
Hertz, D. B. (1964), “Risk analysis in capital investment,” Harvard Business Review, 42 (Jan.-Feb.), 95-106.
Siegel, S (1956), Non-Parametric Statistics for the Behavioral Sciences. New York: McGraw-Hill.
