ArticlePDF Available

The Use of the Decomposition Principle in Making Judgments

Authors:

Abstract

One hundred and fifty-one subjects were randomly divided into two groups of roughly equal size. One group was asked to respond to a decomposed version of a problem and the other group was presented with the direct form of the problem. The results provided support for the hypotheses that people can make better judgments when they use the principle of decomposition; and that decomposition is especially valuable for those problems where the subject knows little. The results suggest that accuracy may be improved if the subject provides the data and the computer analyzes it, than if both steps were done implicitly by the subjects.
Published in Organizational Behavior and Human Performance, 14 (1975), 257-263
The Use of the Decomposition Principle in Making Judgments
J. Scott Armstrong
The Wharton School, University of Pennsylvania
William B. Denniston, Jr.
2nd Lieutenant, U.S. Army
Matt M. Gordon
H.J. Heinz Inc., Pittsburgh, PA
ABSTRACT
One hundred and fifty-one subjects were randomly divided into two groups of roughly equal size.
One group was asked to respond to a decomposed version of a problem and the other group was
presented with the direct form of the problem. The results provided support for the hypotheses that
people can make better judgments when they use the principle of decomposition; and that
decomposition is especially valuable for those problems where the subject knows little. The results
suggest that accuracy may be improved if the subject provides the data and the computer analyzes
it, than if both steps were done implicitly by the subjects.
One of the basic notions behind the scientific method is that a problem should be explicitly "decomposed" or
broken down into a series of subproblems. Solutions are then obtained for each subproblem, and these solutions are
combined as described by Raiffa (1968, p. 271):
The spirit of decision analysis is to divide and conquer: Decompose a complex problem into
simpler problems, get one's thinking straight in these simpler problems, paste these analyses
together with a logical glue . . .
Polya (1948) provided a general discussion on the use of decomposition in problem solving and Hertz (1964)
provides an example of decomposition in the capital investment problem.
With the decomposition principle, one can generally take account of more factors than when making direct or
global judgments. In addition, it would seem possible to analyze information on the various components more
effectively by "computer" than by doing it in one's head. It is also expected that decomposition will improve
accuracy since the errors from the parts should tend to compensate for one another. In view of these advantages, the
following hypotheses are suggested on the use of decomposition relative to the use of direct estimation:
H1: Use of the decomposition principle will lead to more accurate judgments in most situations;
H2: The value of the decomposition principle is greatest where uncertainty is high.
We were unable to find much empirical evidence which was relevant to the above hypotheses. One exception was
provided by Einhorn (1972), who found that decomposition led to better predictions of life span for people with
Hodgkin's disease than could be obtained by global judgments. Below we describe a small study which was
specifically designed to test the two hypotheses.
2
METHODS
Design
In order to compare the direct and decomposed methods, it seemed worthwhile to choose problems in which there
was a "known" answer. It was also important to contrast problems about which the subjects were likely to have good
information with those where they knew little. Five problems were selected; they are presented here in order from
those which were thought to be easy to answer directly, to those which would be difficult (the answers are also
provided here).
1. How many families were living in the United States in 1970? (Answer: 57 million. Source: World
Almanac)
2. How many students do you think dropped out of high school (grades 9-12) in the United States in 1969?
(Answer: 2,690,000. Source: World Almanac)
3. How many packs (rolls) of Polaroid color films do you think were used in the United States in 1970?
(Answer: 51 million. Source: Annual Reports, Polaroid Corporation)
4. How many pounds of tobacco were produced in the United States in 1969? This includes tobacco which
was used for cigarettes, pipes, cigars, and chewing. It also includes tobacco which was exp orted.
(Answer: 1.8 billion. Source: Statistical Abstract of the United States, 1972
5. A Philadelphia radio station and a chewing gum company cosponsored a contest in 1972. The contest
asked all metropolitan Philadelphia high schools to submit as many postcards as they could with the
message "Carefree Sugarless Gum" handwritten on them. There was no limit to number of cards that
could be submitted by a single student or a single school. The school that submitted the most cards was
the location of a free rock concert by the "Grass Roots" band. The contest was heavily promoted. How
many postcards were submitted in the contest? (Answer: 66.5 million. Source: WIBG Radio Station,
Philadelphia, PA).
This ranking of difficulty was verified by asking a convenience sample of six students to rank them according to
how well they thought they could estimate the answers. Their rankings were identical with ours. Mean rankings
were 1.0 for Families, 2.4 for Dropouts, 2.8 for Film, 4.0 for Tobacco, and 5.0 for Contest. It may be noted that the
rankings were very clear-cut except for the Dropouts and Film problems.
The decomposed form of the problems was generated by two of the authors and another professor. None knew the
"true answers" when this step was carried out. The basic strategy was (1) to decompose each problem into as many
parts as were relevant to the answer and could be managed in a reasonable amount of time, (2) to make the parts of
roughly equal importance to the answer, (3) to select parts about which there was relatively good general
information, and (4) to be sure that the final answer could be obtained from the parts by simple, arithmetic
operations. As may be noted, the decomposition also relied heavily upon bounded estimates from 0 to 100%. The
decomposed questions were as follows:
1. a. What was the population of the U.S. in 1970? _____
b. How many people were there in the average family living in the U.S. in 1970? _____
2. a. What do you think was the population of the U.S. in 1969? _____
b. What PERCENTAGE of these people do you think were going to school in 1969 (all grades: kindergarten
through grad school)? _____
c. Of those people going to school in 1969, what PERCENTAGE do you think were in high school (grades
9-12)? _____
d. Of those students in high school in 1969 (of high school age), what PERCENTAGE do you think dropped
out of school in 1969? _____
3
3. a. How many people do you think were living in the U.S. in 1970? _____
b. In 1970 what do you think was the size of the average family living in the U.S.? _____
c. In 1970 what PERCENTAGE of the families do you think owned cameras? _____
e. Of those families that owned cameras, what PERCENTAGE do you think owned Polaroid cameras?
_____
f. What do you think was the average number of packs (rolls) of Polaroid film used per Polaroid camera
owner in 1970? _____
g. What PERCENTAGE of the Polaroid film used in 1970 do you think was color film? _____
4. a. What do you think was the population of the U.S. in 1969? _____
b. What PERCENTAGE of these people do you think smoked:
cigarettes _____
pipes _____
cigars _____
chew tobacco _____
c. Of those people who smoked cigarettes, how many cigarettes on the average (per person per day) do you
think they smoked? _____
d. Of those people who smoked pipes, how many pouches of pipe tobacco on the average (per person per
day) do you think they smoked? _____
e. Of those people who smoked cigars, how many cigars on the average (per person per day) do you think
they smoked? _____
f. Of those people who chewed tobacco, how many bags of chewing tobacco on the average (per person
per day) do you think they chewed? _____
g. On the average, how many cigarettes do you think there are in a pound of tobacco? _____
h. On the average, how many pouches of pipe tobacco do you think there are in a pound of tobacco? _____
i. On the average, how many cigars do you think there are in a pound of tobacco? _____
j. On the average, how many bags of chewing tobacco do you think there are in a pound of tobacco?
_____
k. What PERCENTAGE of the tobacco produced in the U.S. in 1969 do you think was exported? _____
5. A Philadelphia radio station and a chewing gum company recently co-sponsored a contest. The contest asked
all metropolitan Philadelphia high schools to submit as many postcards as they could with the message
"Carefree Sugarless Gum" handwritten on them. There was no limit to the number of cards that could be
submitted by a single student or a single school. The school that submitted the most cards was the location of a
free rock concert by the "Grass Roots" band. The contest was heavily promoted.
a. What is the population of metropolitan Philadelphia? _____
b. Of the people in metropolitan Philadelphia, what PERCENTAGE are in high school (grades 9-12)?
_____
c. Of the high school students in metropolitan Philadelphia, what PERCENTAGE participated in the
contest? _____
d. What was the average number of handwritten postcards submitted by a participant in the contest? _____
It should be noted that subjects answering the direct version of the questionnaire were free to decompose the
problems on their own. However, few subjects wrote any calculations on their questionnaires.
Subjects
Three groups of graduate and undergraduate students from classes at the Wharton School of the University of
Pennsylvania were tested. Group I was composed of 50 students drawn from three separate classes; Group II was
composed of 50 students drawn from two classes; and Group III was composed of 51 students from two classes. In
each class an attempt was made to randomly split the students into two equal subsamples; one subsample was given
the direct questions while the other was given the decomposed questions.
4
Procedure
Each of the authors and one another person was used in the adminis tration of the questionnaires. The
administrators were instructed not to discuss the purpose of the experiment to the subjects prior to, or during the
testing. The instructions and other aspects of the situation varied according to the group.
Group I was given three problems (Dropouts, Film, and Tobacco). The following instructions were read:
We're doing a study on decision making and would like you to help us out. It will take you
less than 15 min. We will be happy to let you know how close your answers are to the true
answers. Please be certain to answer every question. If you are uncertain about any answer,
give your best estimate. If you have any questions, raise your hand. Thank you for your
cooperation.
Group II was given all five problems. In addition, rewards of $15, $10, and $5 were offered for the 3 students (out
of 50 total) who provided the most accurate answers. The following instructions were read:
We're doing a study on decision making and would like you to help us out. It will take you
less than 15 min. There will be cash prizes of $15, $10, and $5 for the three students who
give the best answers. Please be certain to answer every question. If you are uncertain
about any question, give your best estimate. If you have any questions, raise your hand.
Thank you for your cooperation.
Group III was also given all five problems. The procedures for this group were the same as those for group 11
except that minor revisions were made in the wording of the questions to improve clarity, the order of the questions
was reversed, there was no reward, and the following instructions were read:
We are doing a study on decision making. While we cannot explain the purpose of this
experiment at the present time, it is very important that you try to be as accurate as
possible and that you answer every question. You will have 15 min. Please use the
complete 15 min but make sure that you have completed all of the questions in this time.
You will be notified when 10 min remain, then 5 min, then 1 min. Begin immediately.
In summary, it may be noted that certain key factors were varied over the course of the experiment. These included
the setting of the experiment (it was administered at eight different times to eight different classes by four different
administrators); the instructions; the wording of the questions; the ordering of the questions; and the use of a reward
vs. no reward.
Data Analysis
The accuracy ratio was used as the criterion. This was obtained by dividing the "estimate" by the "actual" in cases
where the estimate exceeded actual. If actual exceeded the estimate, then actual was divided by estimate. In simpler
terms, the larger of the two numbers was placed in the numerator and the smaller in the denominator. For example,
if the actual value for a given problem were 10, then answers of both 2 and 50 would have accuracy ratios of 5. The
primary advantage of this ratio was that it eliminated problems with scaling so that the errors in the different
problems might be compared with one another. Another advantage was that the loss function was symmetrical-in
other words, there was no advantage for the subject to try to bias his answers on either the high or low side.
RESULTS
The results from Groups I, II, and III are presented in Table 1. The subjects providing the decomposed estimates
did worse than those providing direct estimates on only one of the 13 comparisons. Furthermore, on 9 of the 13
comparisons the decomposed version was significantly more accurate (significant at .05 level using the median test
from Siegel, 1956). The estimates obtained using decomposition were also less likely to have large errors. For
5
example, for all groups combined, 64% of the direct estimates for the Tobacco problem were off by a factor of ten or
more, while only 20% of the decomposed estimates for this problem were off by such a degree. These results
support hypothesis H1 that use of the decomposition principle leads to more accurate estimates. There was' no
evidence that accuracy was affected by a reward. Follow-up dis cussions with each group also led to mixed reactions
as to whether the reward had any effect and, if so, what would be the direction of the effect. In any event,
decomposition led to significant improvements with or without the use of a reward.
TABLE 1
Median Accuracy Ratio: Decomposed vs. Direct Questions
Question
(1)
Original rank
(2)
Direct version
(n = 25)
(3)
Decomposed version
(n = 25)
Difference
Group 1
Dropouts
Film
Tobacco
2
3
4
5.3
21.1
103.2
2.2
2.1
2.1
3.1
190
101.1
Group 2
Families
Dropouts
Film
Tobacco
Contest
1
2
3
4
5
1.3
5.4
10.2
18.0
831.3
1.3
2.3
7.0
7.4
147.8
0.0
3.1
3.2
10.6
683.5
Group 3
Families
Dropouts
Film
Tobacco
Contest
1
2
3
4
5
1.4
5.9
5.1
9.0
351.0
1.2
6.0
2.7
5.7
184.7
0.2
0.1
2.4
3.3
166.3
The rankings of differences for Groups I and II were in perfect agreement with the hypothesis (see columns 1 and
4 in the table). While the number of questions for Group I was too small to allow for a test of statistical significance,
the rankings in Groups II and III were each significant at the .05 level (using Spearman Rank Correlation from
Siegel, 1956). Thus, the results also support H2--i.e., that the value of decomposition is greatest where the subject's
knowledge is poorest.
FURTHER RESEARCH
Further research would be useful in trying to better define how the decomposition should be carried out. For
example, does it help to use bounded questions? To what degree should the problem be decomposed? What role
does the scaling (units) of the answers have upon the results? Is it better to have the subject decompose the problem
in order to make better use of his own information? We did some exploratory work on this last question by
comparing 11 subjects who decomposed two problems with 11 subjects who were provided with a decomposed ver-
sion. The self-decomposition was poorer on each question, but the differences were not significant.
REFERENCES
Raiffa, H. (1968), Decision Analysis. Reading, Mass.: Addison-Wesley.
Polya, G. (1948), How To Solve It. Princeton, N.J.: Princeton University Press.
6
Einhorn, H. J. (1972), “Expert measurement and mechanical combination,” Organizational Behavior and Human
Performance, 7, 86-106.
Hertz, D. B. (1964), “Risk analysis in capital investment,” Harvard Business Review, 42 (Jan.-Feb.), 95-106.
Siegel, S (1956), Non-Parametric Statistics for the Behavioral Sciences. New York: McGraw-Hill.
... 377Vgl. Libby; Libby 1989, S. 729-747;Armstrong;Denniston Jr; Gordon 1975, S. 257-263. 378 Vgl. ...
... 377Vgl. Libby; Libby 1989, S. 729-747;Armstrong;Denniston Jr; Gordon 1975, S. 257-263. 378 Vgl. ...
Thesis
Die Digitalisierung in der Geschäftswelt mit ihren exponentiell steigenden Datenmengen erhöht die Anforderungen an die Verarbeitung und effiziente Aufbereitung entscheidungsrelevanter Daten ununterbrochen. Durch die arbeitsteilig bedingte Spaltung zwischen Berichterstellenden und -empfangenden entstehen kognitive Verzerrungen und verschiedene Verhaltenswirkungen beim Informationstransfer in Form von Management Reporting. Behavioral Management Accounting untersucht diese Verhaltenswirkungen, die sich in deklarierte, manipulative, reaktive und versteckte aufteilen. Diese Untersuchung widmet sich den kognitiven Verzerrungen sowie deren Auslösern. Unter Verwendung der Methoden qualitativer Forschung wird in diesem Zusammenhang in einer explorativen Herangehensweise untersucht, inwieweit International Business Communication Standards (IBCS) als Debiasing-Instrument den Heuristiken in Management Reporting entgegenwirken können. EN: Digitization in the business world, with its exponentially growing data volumes, is increasing the demand on the processing and efficient preparation of decision-relevant data. During the transfer of information in the form of Management Reporting, various cognitive and behavioral effects arise due to a division of work between the message creator and recipient. Behavioral Management Accounting examines these behavioral effects, which are divided into declared, manipulative, reactive and hidden effects. This work is dedicated to the hidden behavioral and cognitive effects and their triggers. Employing the method of qualitative research, an exploratory approach is used to investigate to what extent the International Business Communication Standards (IBCS) as a debiasing instrument can counteract the heuristics in Management Reporting.
... Structuring involves defining the event under consideration to minimise ambiguity in the questions and to explore how an expert thinks about the quantity for which probability judgement is to be elicited. The aim is to manage possible cognitive bias by simplifying the complex task of assigning probabilities by disaggregating the quantity of interest into more elementary variables (Armstrong et al. 1975). However, the unpacking principle (O'Hagan et al. 2006), also known as subadditivity (Tversky and Koehler 1994), may be the consequence. ...
Chapter
The elicitation of subjective probabilities from experts can be critical in determining a course of action when making decisions under uncertainty. A sound process to elicit probabilistic judgement is necessary to ensure that good quality data are used to inform the decision-making, as well as to provide protection to those accountable for the consequences of the determined actions. We synthesise the characteristics of a good elicitation process by critically reviewing those advocated and applied. We compare the processes inherent in the guidance produced by two professional bodies to exemplify the manner in which the characteristics manifest themselves in practice. We examine whether standardisation is meaningful given the maturity of processes for the elicitation of subjective probability.
... This stage is concerned with encoding the expert's uncertainty on the quantity of interest as a probability. Prior to eliciting these quantities training experts on probability and providing relevant information for discussion should be conducted to minimize the presence of potential biases (Tversky & Kahneman 1974;Armstrong et al. 1975). In particular, this can address biases such as anchoring (Tversky & Kahneman 1974), i.e. when the evaluation is conditioned by an initial assessment, and availability (Kahneman & Tversky 1973), i.e. when the evaluation is based on the ease with which relevant instances come to mind. ...
Article
Full-text available
Bayesian networks support the probabilistic failure analysis of complex systems, e.g. dams and bridges, needed for a better understanding of the system reliability and for taking mitigation actions. Bayesian networks are useful in representing the interactions among system components graphically, while the quantitative strength of the interrelationships between the variables is measured using conditional probabilities. However, due to a lack of objective data it often becomes necessary to rely on expert judgment to provide subjective probabilities to quantify the model. This paper proposes an elicitation process that can be used to support the collection of valid and reliable data with the specific aim of quantifying a Bayesian Network, while minimizing the adverse impact of biases. To illustrate how this framework works, it is applied to a real-life case study regarding the safety of the Mountain Chute Dam and Generating Station, which is located on the Madawaska River in Ontario, Canada.
... The theory of survey satisficing (Krosnick 1991) suggests that branching may be a preferable format for most questions that use a bipolar response format, and other research fits with this theory to support branching. Armstrong, Denniston, and Gordon (1975) demonstrated that complex judgments are made more accurately when they are decomposed into easier component tasks. When applied to a bipolar construct, the branching format initially presents respondents with one easy task, to state the valence of their opinion, followed by a second easy task, to state the intensity of their opinion. ...
Article
This article reports the results of an experiment comparing branch, grid, and single-item question formats in an internet survey with a nationally representative probability sample. We compare the three formats in terms of administration time, item nonresponse, survey breakoff rates, response distribution, and criterion validity. On average, the grid format obtained the fastest answers, the single-item format was intermediate, and the branch format took the longest. Item nonresponse rates were lowest for the single-item format, intermediate for the grid, and highest for branching, but these results were not statistically significant when modeling the full experimental design. Survey breakoff rates among the formats are not statistically distinguishable. Criterion validity was weakest in the branching format, and there was no significant difference between the grid and single-item formats. This evidence indicates that the branching format is not well suited to internet data collection and that both single-item and short, well-constructed grids are better question formats.
... There is an increasing amount of empirical evidence to show that, indeed, this principle holds 23,24,25 . ...
... There is an increasing amount of empirical evidence to show that, indeed, this principle holds 23,24,25 . ...
Article
Bayesian networks are a popular class of multivariate probabilistic models as they allow for the translation of prior beliefs about conditional dependencies between variables to be easily encoded into their model structure. Due to their widespread usage, they are often applied to spatial data for inferring properties of the systems under study and also generating predictions for how these systems may behave in the future. We review published research on methodologies for representing spatial data with Bayesian networks and also summarize the application areas for which Bayesian networks are employed in the modeling of spatial data. We find that a wide variety of perspectives are taken, including a GIS-centric focus on efficiently generating geospatial predictions, a statistical focus on rigorously constructing graphical models controlling for spatial correlation, as well as a range of problem-specific heuristics for mitigating the effects of spatial correlation and dependency arising in spatial data analysis. Special attention is also paid to potential future directions for integration of Bayesian networks with spatial processes.
Article
We study how the menu of contracts presented to a decision maker—including contracts she may be precluded from choosing—affects her choice of remunerative but risky actions relative to lower paying, less risky alternatives. We do this through a series of laboratory experiments modeled after the loan repayment options offered to U.S. student borrowers, analyzing borrowers’ task (career) choices in settings that vary the menu of available and unavailable loan repayment plans and knowledge of unavailable options. In these experiments, we observe behavior that is inconsistent with predictions from standard economic models in which agents can easily make complex decisions and each alternative in a choice set is evaluated independently of other potential options. Instead, we provide evidence that expanding the menu of choices or making an agent aware of choices that she has been denied can affect how a contract is valued. Our empirical findings are most consistent with behavioral models that allow for anticipated regret over a choice that turns out to be suboptimal ex post or preferences for simplicity and gratitude for being unburdened from having to make a choice. This paper was accepted by Yan Chen, decision analysis.
The expert can and should be used as a provider of input for a mechanical combining process since most studies show mechanical combination to be superior to clinical combination. However, even in expert measurement, the global judgment is itself a clinical combination of other judgmental components and as such it may not be as efficient as a mechanical combination of the components. The superiority of mechanically combining components as opposed to using the global judgment for predicting some external criterion is discussed. The use of components is extended to deal with multiple judges since specific judges may be differentially valid with respect to subsets of components for predicting the criterion. These ideas are illustrated by using the results of a study dealing with the prediction of survival on the basis of information contained in biopsies taken from patients having a certain type of cancer. Judgments were made by three highly trained pathologists. Implications and extensions for using expert measurement and mechanical combination are discussed.
Chapter
The abstract for this document is available on CSA Illumina.To view the Abstract, click the Abstract button above the document title.
Decision analysis How to solve it
  • H Raiffa
  • G Polya
Raiffa, H. Decision analysis. Reading, Mass.: Addison-Wesley, 1968. Polya, G. How to solve it. Princeton, N.J.: Princeton University Press, 1948.