Content uploaded by Kevin C.K. Lam
Author content
All content in this area was uploaded by Kevin C.K. Lam on Nov 27, 2014
Content may be subject to copyright.
Implementing the Balanced Scorecard
Using
the Analytic Hierarchy Process & the Analytic Network Process
Lawrence C. Leung 1*, Kevin C. Lam2** and D. Cao3
1E-mail: lawrence@baf.msmail.cuhk.edu.hk
2E-mail: kevinl@baf.msmail.cuhk.edu.hk
3E-mail: Dong_Cao@uqtr.ca
1,2 Faculty of Business Administration, The Chinese University of Hong Kong
3Department of Industrial Engineering, University of Quebec at Trois-Rivieres
May 2005
2
Implementing the Balanced Scorecard
Using
the Analytic Hierarchy Process & the Analytic Network Process
Abstract The Balanced Scorecard (BSC) is a multi-attribute evaluation
concept that highlights the importance of non-financial attributes. By
incorporating a wider set of non-financial attributes into the measurement
system of a firm, the BSC captures not only a firm’s current performance, but
also the drivers of its future performance. Although there is an abundance of
literature on the Balanced Scorecard (BSC) framework, there is a scarcity of
literature on how the framework should be properly implemented. In this
paper, we use the Analytic Hierarchy Process (AHP) and its variant the
Analytic Network Process (ANP) to facilitate the implementation of the BSC.
We show that the AHP and the ANP can be tailor-made for specific situations
and can be used to overcome some of the traditional problems of BSC
implementation, such as the dependency relationship between measures and
the use of subjective versus objective measures. Numerical examples are
included throughout.
[Keywords: AHP, ANP, Balanced Scorecard, Performance
Measurement.]
3
1. INTRODUCTION
It is well recognised in the literature that performance measurement and reward
systems should incorporate both financial and non-financial measures (Dyson, 2000; Banker
and Datar, 1989). However, firms predominantly adopt the traditional approach of relying
exclusively on financial measures. A number of studies (Smith and Goddard, 2002;
Brancato, 1995; Eccles and Pyburn, 1992; Maskell, 1992) have illustrated that the traditional
financial measures are too historical and ‘backward looking’, lack the predictive ability to
explain future performance, reward short-term or incorrect behaviour, are too aggregated and
summarised to be able to guide managerial action, and do not consider the relevant
intangibles.
The seminal work by Kaplan and Norton provides a multi-criteria framework within
which performance evaluation can be conducted. Termed the Balanced Scorecard (BSC),
this multi-attribute evaluation concept highlights the importance of non-financial attributes.
By incorporating non-financial attributes into the measurement system of a firm, the BSC
seeks to create a wider set of measures that capture not only a firm’s current performance, but
also the drivers of its future performance.
Kaplan and Norton (1992) categorise evaluation measures into four perspectives:
financial, customer, internal business process, and learning and growth. With the BSC,
companies can evaluate their top managers in terms of their effectiveness in creating value
for customers, developing internal capabilities, and investing in the people and systems that
are necessary to improve future performance. The BSC captures critical value-creation
activities that escape traditional income statements and balance sheets, but maintains an
interest in short-term financial performance.
In this paper, we address two important and related aspects in the implementation of
BSC: the handling of dependency between perspectives – especially those of a subjective
4
nature, and the determination of the contribution of the respective perspectives. It has been
well recognized that dependency between perspectives (attributes) could exist in a BSC. We
show that with the ANP, the dependency between attributes, which can be objective or
subjective, can be explicitly incorporated. The contribution of the respective perspective can
also be properly assessed by utilizing either the AHP or the ANP framework. Although
previous studies have used the AHP to study performance measurement − including assisting
the implementation of the Balanced Scorecard (e.g. Liberatore and Miller, 1998) − these
studies did not take into account the dependencies among the performance measures. In the
remainder of the paper, BSC implementation issues are first discussed in Section 2. In
Section 3, we present how the AHP is used when dependency between attributes can be
implicitly assessed. In Section 4, the ANP is used to explicitly assess the dependency effects.
Section 5 is devoted to the handling of time-dependency issues. Finally, conclusions are
drawn and future research is suggested in Section 6.
2. BSC IMPLEMENTATION ISSUES
Although the conceptual framework of the BSC has been widely accepted in the
business community, the proper method of implementing the framework remains an issue.
Table 1 depicts a typical implementation scenario of the BSC scheme.
Table 1. A Simple Example of BSC Implementation
Performance Bonus Incentive Actual Incentive
Measure Budget Schedule Performance Earned
Performance %@Budget
Financial $150 20%+ .................. 100% 18% $100
- Return on 16%-20% .......... 66.7%
Capital 12%-16% .......... 33.3%
Employed Below 12% ............. 0%
1 in:
Customer $60 1,000+ ................ 100% 876 $20
5
- Product Returns 900-999 ............. 66.7%
800-899 ............. 33.3%
Below 800 .............. 0%
Internal $45 9%+ .................... 100% 11% $45
- Cycle Time 6%-9% .............. 66.7%
Reduction (%) 3%-6% .............. 33.3%
0%-3% ................... 0%
Learning & Growth $45 Below 5% ........... 100% 7% $30
- Voluntary 5%-8% .............. 66.7%
Employee Turnover 8%-12% ............ 33.3%
Above 12%……….0%
Total Budget Total Actual
Incentive $300 Incentive $195
_________________________________________________________________________________________
___
In this scenario there are four measures, with one measure for each perspective. A list
of the measures that are commonly used in a BSC system is provided in the Appendix. In
practice, companies that implement a BSC use between ten and twenty-five measures
(Kaplan and Atkinson, 1998). Tangible ‘proxies’, such as defect and absenteeism rates, are
used to capture the intangible attributes. Whenever tangible proxies cannot be obtained,
survey scores, such as customer satisfaction surveys, are used. The bonus budget column
represents the relative weighting of each perspective. The incentive schedule defines the
percentage distribution of rewards for different levels of performance. Actual performance is
compared with the incentive schedule to determine the corresponding reward. In our simple
example, the weights of the measures and sub-measures are assumed to be known a priori.
However, the question remains as to how they should be determined. Kaplan and Norton
(1996) and Ittner and Larcker (1998), among others, have raised the following issues.
2.1 Relationship between the criteria weights
An important aspect of any multiple criteria decision scheme is that the appropriate
weight must be placed on each measure or criterion. However, the BSC framework does not
6
provide guidance as to how these weights should be computed. There are many ways, or
plans, to distribute the reward. Some of these plans allow bonuses to be paid even when
performance is ‘unbalanced’ (i.e., when there has been over-achievement in some areas but
under-achievement in others). Other plans require a minimum level (hurdle) to be attained in
each perspective before bonuses can be paid. The relationships between the perspectives also
complicate the determination of the weight of the attributes. The BSC acknowledges the
presence of dynamic relationships among the perspectives, which means that the importance
of one perspective cannot be determined without knowing the effects of the relationships
between the perspectives. It is important that the proper weights be determined for the
criteria to avoid situations in which a manager is inappropriately rewarded or penalised.
2.2 Time-dependent relationships
Relationships among the criteria can also exist across time. The BSC literature makes
a clear distinction between two types of performance indicators: the outcome measures and
the performance drivers. Outcome measures are lagging indicators that show the final
results, whereas performance drivers are leading indicators of the strategy of the business unit
and enable the business unit to achieve short-term operational improvements. However,
these measures fail to reveal whether the operational improvements have been translated into
expanded business and eventually into enhanced financial performance. There are well-
recognised time dependencies between performance drivers and outcome measures (Kaplan
and Norton, 1996). Moreover, a firm’s product and organisational life-cycle may affect its
business strategy and the design of its management control system (Simons, 1999). How to
incorporate these dependencies and tailor a firm’s scorecard accordingly has not been
addressed in the literature.
2.3 Subjective versus objective measures
7
There has been considerable debate as to whether firms should use objective or
subjective measures in performance appraisals. Analytical studies (e.g., Baiman and Rajan,
1995) indicate that subjective compensation plans may be superior, because they allow firms
to exploit non-contractible information that might otherwise be ignored in formula-based
plans. Moreover, objective surrogate measures often inaccurately reflect intangible criteria.
Nonetheless, subjective evaluations are vulnerable to accusations of favouritism or other
kinds of abuses, whereas objective measures may be perceived to be more fair and
transparent. Another critical consideration is how the weights of the subjective and objective
criteria should be determined if both types of criteria are used in the BSC.
3. MODELLING THE BSC USING THE AHP
Although there is an abundance of literature on the BSC framework, there is a scarcity
of literature on how this framework should be properly implemented. In this paper, we
implement the BSC using the Analytic Hierarchy Process and its variant, the Analytic
Network Process. We show that the AHP and the ANP can be used to address some of the
traditional implementation problems of the BSC, such as the dependency relationship
between the attributes and whether or not subjective or objective measures should be used.
Developed by Saaty (1980), the AHP is a widely used multi-criteria decision theory that can
incorporate both objective and subjective information. When relationships exist between the
criteria themselves and between the criteria and the sub-criteria, the ANP must be used
(Saaty, 1996).
The AHP first breaks the problem into a hierarchy of attributes and sub-attributes.
Typically, the overall goal is at the top, and the choice alternatives are at the very bottom.
The relative importance of the sub-attributes with respect to a given attribute is determined
by using ratio scales and paired comparisons. The methodology then respectively aggregates
8
the weights of the sub-attributes at a lower level to form the weights at the higher level. The
final result is the relative importance of each alternative with respect to the overall goal. An
important feature of AHP is its capability to evaluate intangibles, which results from its use
of relative preferences and ratio scales. In the early 1980s, there was considerable debate
regarding the rank reversal behaviour of the AHP, because it might reverse the rankings of
the alternatives when new alternatives are added or when existing alternatives are deleted
(Belton and Gear, 1983; Harker and Vargas, 1990; Leung and Cao, 2001). However, as this
research is solely concerned with the distribution of bonuses, rather than the ranking of
alternatives, rank reversal is not a practical issue here.
Several works have applied the AHP methodology to study performance measurement
(Chan and Lynn, 1991; Rangone, 1996; Suwignjo et al., 2000). Liberatore and Miller (1998)
have used the AHP to implement a BSC where both objective and subjective performance
measures are considered. However, these earlier papers did not explicitly take into account
the dependencies among attributes. When the decision-making process involves attributes
that have a dependency relationship, the problem should be modeled as an ANP. This is
especially important in a BSC situation where such dependency is pervasive. This study
proposes that the AHP and its variant the ANP is a robust approach to facilitate the
implementation of the BSC.
3.1 The basic AHP model
We first formulate the BSC as a simple AHP model, as it is shown in Figure 1. The
tree-like hierarchy connotes the distribution of importance from the overall goal down to the
perspectives, their measures and sub-measures, and then the ratings. The objective is to first
determine the schedule of the bonus distribution, and then to determine the amount that
should be apportioned to the manager based on actual performance.
9
R2
R1RN
P2 PM
P
1
GOAL
P
11 P12
Perspectives
Levels:
Measures
Ratings
.
.
.
. . .
. . .
…P21 P22 …PM1 P
M2 …
Sub-measures
.
.
.
Figure 1. The Basic AHP Model for the BSC
We let w=(w1,…,wM)T be the relative weight distribution of the perspectives with
respect to the overall goal; the matrix A1 = [A11, A12,…,A1M] be the relative weight
distribution of the measures with respect to the perspectives, where A1j is the relative weight
vector of the measures with respect to the perspective j; the matrices Ai = [Ai1, Ai2,…,Ai…, ], i
= 2,…, k-1 be the relative weight distribution of the sub-measures at level i with respect to
those at level i-1, where Aij is the relative weight vector of the sub-measures at level i with
respect to the sub-measure j at level i-1 ; and the matrix Ak = [Ak1, Ai2,…,Ak…, ], be the
relative weight distribution of the ratings with respect to the sub-measures at level k-1. The
relative importance of the measures or sub-measures at level k-1, with respect to the overall
goal, is determined by
ZGM = Ak-1….A1w (1)
The relative importance of the ratings to the overall goal is determined by
ZGR = AkZGM (2)
10
Here ZGR is the percentage of the total bonus budget allocated to the respective perspectives.
Details of determining Ak , Ak-1,….A1, and w using the AHP methodology can be found in
Saaty (1980) and will be omitted here.
Next, we determine the actual amount of bonus that would be given to a manager
based on a given set of performance ratings. Let b be the respective amount of bonus budget,
in terms of dollars, corresponding to individual ratings. The following relationship holds.
b =
ZGR (3)
where
is the sum of all the respective bonuses. Note that the decision maker needs to
provide b.
The actual amount of bonus payment to the manager is
B = bTR*ZGM (4)
where R is the matrix representing the actual performance of the manager (in terms of the
relative importance of the ratings with respect to the measures or sub-measures at level k-1).
As an illustrative example, refer back to the simple example in Table 1 (see also
Figure 2).
SatisfactoryGood
Excellent Poor
Internal Customer
Learning
&Growth
Financial
GOAL
Figure 2. The AHP Example for the BSC
Here, the four perspectives are at a level below the overall goal. Only one criterion
level is needed, as there is only one measure for each perspective and no sub-measures. In
11
addition, each level of the incentive schedule corresponds to each rating, that is, excellent,
good, satisfactory, and poor. Assuming that the decision-makers have gone through the AHP
process of pairwise comparisons, consistency test, weight determination, etc (see Saaty
(1980) for details in the AHP process), the weights of the perspectives (in percentage of total
bonus budget) is ZGM = w = (1/2, 1/5, 3/20, 3/20)T. The relative distribution of the ratings per
perspective are A11 = (1/2, 1/3, 1/6, 0)T, A12 = (1/2, 1/3, 1/6, 0)T, A13 = (1/2, 1/3, 1/6, 0)T, and
A14 = (1/2, 1/3, 1/6, 0)T. In this example, note that the dimensions of the ratings for each
perspective are the same. When the dimensions are different, zero values would be placed in
the empty entries of the matrix A correspondingly. The aggregate amount of bonus
corresponding to the four ratings respectively are: excellent ($150+60+45+45)=$300 (this is
the maximum amount the manager can obtain; i.e., the total bonus incentive); good
($100+40+30+30)=$200; satisfactory ($50+20+15+15)=$100; and poor ($0+0+0+0)=$0.
That is, b =($300, 200, 100, 0)T , which has the following relationship b =
AkZGM and
=
(300+200+100+0)=600.
In this example, the manager’s actual performance (good, satisfactory, excellent,
good) is measured for the respective perspectives. We have objective data on the relative
preference of the respective ratings. The manager’s respective relative importance in the
ratings is simply R11 = (0, 1, 0, 0)T, R12 = (0, 0, 1, 0)T, R13 = (1, 0, 0, 0)T, and R14 = (0, 1, 0, 0)T.
If subjective comparisons have been sought, then the relative preference is likely to be a
vector of fractional values. The actual bonus payoff would then be
B = bTR*ZGM
195$02013045$
20/3
20/3
5/1
2/1
0000
0010
1001
0100
0
100
200
300
T
12
Essentially, the manager is rated slightly below ‘good’ overall.
It should be pointed out that unless the expectation for an individual manager varies,
there is no need to repeat the weight determination for the perspectives for each manager’s
evaluation. However, such weights can change over time, and may have to be recalculated
periodically. Additionally, the AHP implementation can easily incorporate a bonus with
hurdle requirements. For example, if the reward system is such that no bonus will be
awarded when a manager has a rating of ‘poor’ in one or more perspective, then a large
penalty (negative value) can be attached to the bonus payment.
4. MODELLING THE BSC USING THE ANP
Any two attributes can be considered to be independent if they are unrelated, such as
the colour and the smell of food. In the preceding basic AHP model, it is assumed that the
decision maker is capable of factoring in all of the interactions, and can directly provide
preferences between the perspectives. Conversely, any two attributes can be considered to be
dependent if they are related, such as taste and the temperature of food. In this case, the
determination of a preference between the two attributes requires an assessment of the level
of dependency between them.
4.1 A BSC model with dependency
When the decision-making process involves attributes that have a dependency
relationship, the problem should be modelled as an ANP (Saaty, 1996). Here, we formulate
the BSC problem as an ANP, as is shown in Figure 3. A collection of similar attributes is
referred to as a cluster, and the perspectives themselves form a cluster. The attributes of each
measure or sub-measure form a cluster, and there are K such clusters. The ratings also form a
cluster. The dependency relationship between the attributes within a cluster is called
13
innerdependency, which is denoted by a directed loop for the cluster. A two-way
dependency relationship between attributes in two different clusters is called
interdependency, which is denoted by a two-way directed arc between the clusters.
To solve this BSC formulation, we need to separate the problem into two parts. In the
first part the weights of the perspectives and their measures and sub-measures are
determined, and in the second part the weights of the ratings are determined. These two
aspects need to be separately addressed because the determination of the two sets of weights
is independent.
R2 R1RN
P2 PM
GOAL
...
...
C1
CK
Ci
...
P1
Figure 3. Modelling a BSC with Dependency Using the ANP
Determining the weights of the perspectives and their measures and sub-
measures. The first step in determining such weights is to construct a supermatrix as
follows:
Goal P C1 … Ci … CK
Goal 0 0 0 0 0
P
w APP 0 0 0
C1 0 APC1 AC11 AC1i AC1K
14
W = : : : : : :
Ci
0 APCi ACi1 ACii A
CiK
: : : : : :
CK 0 APCK ACK1 ACKi ACKK
where P is the cluster of perspectives, w is the relative importance vector of the perspectives
(P1,…,Pm) to the goal, Ci are the measures (or sub-measures) that are grouped in cluster i,
i=1,…,K, APP is the relative importance matrix of the perspectives to each other
(innerdependence), APCi is the relative importance matrix of the measures (or sub-measures)
in cluster i to the perspectives, and ACij is the relative importance matrix of the measures (or
sub-measures) in cluster i to those in cluster j.
For each cluster (leading cluster) that is directly affected by several other clusters, we
need to determine the relative importance of these clusters to the leading cluster. Within the
supermatrix framework, this means the determination of the relative importance of a column
of clusters to the leading cluster of that column. The weights of the relative importance of
the clusters (of a column) are used to normalise the elements of the respective individual
clusters in that column. This also maintains the unity property of the columns of the
supermatrix. Raising the supermatrix W to an infinite power (in practice, to a large power),
we have W
, the elements of which are stable. More details about this approach can be found
in Saaty (1996). The relative weights of the perspectives and their measures and sub-
measures with respect to the overall goal are given in the first column of the converged
matrix W
.
Determining the weights of the ratings. The determination of the relative
importance of the ratings to the goal can be obtained by
15
ZGR= ACR* ZGM (5)
where ACR is the matrix that represents the relative importance of the ratings to their related
measures or sub-measures, and ZGM is the vector that represents the relative importance of the
measures or sub-measures to the ratings with respect to the overall goal, where ZGM is in the
first column of the converged matrix W
.
Let
be the total allotted bonus. The respective bonus that corresponds to the respective
ratings is determined in a similar way to (2) by the following equation:
b =
ZGR=
ACR* ZGM (6)
The actual bonus payment to the manager can therefore also be determined by Equation (4).
4.2 An illustrative example
As an example, we again consider the BSC example in Table 1 (see also Figure 4).
For ease of reference, the four perspectives are organised into two groups: present
performance and future performance. Present performance is simply the financial
perspective, and future performance has three sub-attributes of learning and growth, internal
process and customer. The present (financial) performance and the future performance are
assumed to be unrelated. The future performance depends on the three measures, which are
dependent on one another. Unlike the AHP BSC model, we cannot determine the relative
importance of the perspectives with reference to the overall goal alone. The relative
importance of the attributes must be determined with additional consideration of their impact
on each another.
16
PoorSatisfactoryGood
Excellent
Future
Present
Goal
Learning
& Growth Customer
Internal
Process
Figure 4. A BSC Example Problem with Innerdependency
In the ANP, as in the AHP, a paired comparison of the relative importance of the sub-attributes
to the leading attribute is conducted. However, the presence of innerdependencies complicates
the pairwise comparison process. We perform pairwise comparisons for all of the perspectives
within the future cluster (learning and growth, internal, customer). First we determine the
relative importance of one perspective to another for each perspective in the cluster.
Essentially, we ask such questions as ‘with respect to learning and growth, is internal more
important than learning and growth, and by how much?’ The following shows an example of
the pairwise comparison matrices for the three innerdependent future perspectives.
L&G
Int
Cus
L&G L&G Int Cus
2
1/3
1
6
1
3
1
1/6
1/2 L&G
Int
Cus
Int L&G Int Cus
2/3
5/3
1
2/5
1
3/5
1
5/2
3/2 L&G
Int
Cus
Cus L&G Int Cus
4/3
1
1
4/3
1
1
1
3/4
3/4
For example, with respect to internal, learning & growth is considered to be 3/2 times more
important than customer, and with respect to customer, internal and learning & growth are
17
equally important. The principal eigenvectors, which is a common method to determine the
weights of the attributes of the three matrices, are:
.6
.1
.3
.2
.5
.3
.4
.3
.3
L&G
Int
Cus
L&G Int Cus
.
Pairwise comparisons for the three future perspectives with respect to future performance, and
for present and future performances with respect to the overall goal are as follows; the principal
eigenvectors (EV) are also included.
Goal
Fin
Fut
Fin Fut
11
11
L&G
Int
Cus
Fut L&G Int Cus
1
3/4
1
3/4
1
4/3
1
4/3
1
EV
.5
.5 .3
.4
.3
EV
Using the eigenvectors from the matrices, we form the supermatrix
Goal
Fin
L&G
L&G
Goal Fin
Int
Int
Cus
Cus
000 0 0
Fut
Fut
0
.5
0
0
0
.5
1
0
0
0
0
0
.3
.4
.3
0
0
.6
.1
.3
0
0
.2
.5
.3
0
0
.4
.3
.3
0
Determining the composite weights. To obtain the weights of the financial
perspective and the three future perspectives with respect to the overall goal, the supermatrix
is raised to a large power (e.g., 25 times). The weights of the four perspectives are the
corresponding values in the first column of the resulting matrix. Here, we arrange ZGM as
(Fin, Int, Cus, L&G) = (.5, .2, .15, .15). Similar to the AHP case, the decision maker can now
18
determine ZGR= ACR* ZGM, followed by the bonus distribution as determined by Equations (6)
and (4). Note also that the criteria weights are contrived to be identical for both the basic
AHP case and the ANP case. In the basic AHP case, the decision maker is able to directly
assess the innerdependent effects without the need to use an ANP model to capture the
effects. It seems reasonable to suggest that the more complex the interactions are, the greater
the need to utilise the ANP.
4.3 BSC with total interaction and time-dependent perspectives
The preceding ANP example does not address the relationship between present
financial performance and future performance. If the decision environment is such that the
present and future performance are related, then the ANP model will be as shown in Figure 5,
in which all four of the perspectives are interacting. This is, in fact, one of the original
versions of the BSC (Kaplan and Norton, 1996). The solution procedure is similar to the
preceding example.
Poor
Satisfactory
Good
Excellent
Goal
Financial
Learning
&
Growth Customer
Internal
Process
Figure 5. Modelling the Total Interaction in the BSC Using the ANP
19
Time-dependent perspectives. Instead of treating the non-financial perspectives as a
group of factors that describe the general future, we can explicitly characterise the
importance of the perspectives as being time variant. Essentially, to determine the present
weight of the measures for the individual perspectives, we need to incorporate the
dependency relationship between the present measures and the future measures. We now
illustrate this aspect with a comprehensive example.
5. A TIME-DYNAMIC BSC
In this section, we provide a comprehensive example in which there are relationships
among the criteria across time, as well as subjective judgements and objective evaluations.
Consider a subsidiary of a consumer electronics manufacturer. This subsidiary produces a
single household appliance and is about to launch a new design. According to the company’s
market analysis, the demand will not be very high for the first two years (introduction stage).
There will be design changes, and the product will maintain a high price, but will be of good
quality. The competition will be based on product quality and should be fierce. In the
second two years (growth stage), the product design should undergo very few changes,
demand will grow at a high rate and sales will climb rapidly. There will be few competitors
and the high price may not be sustainable. Some improved features should then be added to
stay ahead of the competition. The planning horizon is four years, as information beyond that
is not very reliable.
The company intends to implement a BSC system to evaluate the performance of the
manager of this subsidiary. It has been decided that the performance measures will hinge on
the capital employed, customer satisfaction, cycle time reduction and employee satisfaction.
The measure for the customer perspective, customer satisfaction, is a subjective measure, and
is believed to be the most critical measure for the company during the introduction stage.
During the growth stage, cycle time reduction and employee satisfaction will become
20
significantly important. Customer satisfaction is also indirectly affected by cycle time
reduction and employee satisfaction (another subjective measure). These measures are
regarded as the principal indicators of future financial performance.
5.1 A time-dynamic BSC example
Because of the complexities of the dependency relationships, the decision makers
seek to determine the weights of these measures using the ANP model, as is shown in Figure
6. In this example, the overall evaluation is distributed to three principal areas: the present
ROCE, the performance of the subsidiary during the introduction stage and the performance
of the subsidiary during the growth stage. The latter two performances are inferred from the
manager’s current performance in the three inter-related measures of customer satisfaction-I,
cycle time reduction-I, and employee satisfaction-I. These measures are directly related to
the performance of the subsidiary during the introduction stage, and are also related to the
measures of the growth period, which in turn reflect the performance of the subsidiary during
the growth stage.
21
Present
ROCE
Goal
Intro
Growth
PoorSatisfactory Good
Excellent
Customer
Satisfaction-G Cycle Time
Reduction-G Employee
Satisfaction--G
Customer
Satisfaction-I Cycle Time
Reduction-I Employee
Satisfaction--I
Figure 6. An ANP Example with Time-dependent Effects.
5.2 Numerical illustration
Pairwise comparisons are performed as in the previous sections. The principal
eigenvectors are arranged in a supermatrix as follows:
GroGoal Fin Cus-G Int Cycle-G Emp-G Cus-I Cycle-I Emp-I
Goal
Fin
0
.4
0
0
.2
.4
0
0
0
1
0
0
0
0
0
0
0
.2
.5
.3
0
.4
.3
.3
000
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
.15
.2
.15
.25
.1
.1
.15
.15
.05
.1
.3
Gro
Int
Cus-G
Cycle-G
Emp-G
Cus-I
Cycle-I
Emp-I
00.2.1
00.2.3
0
0
0
0
.4
.3
.3
0
0
0
0
.2
.5
.3
0
0
0
0
.1
.3
.6
0
0
0
.3
.1
.1
0
0
0
0
0
0
0
0
0
0
0
0
.
First, we need to determine the relative importance of the introduction stage and the growth
stage with respect to the growth stage. That is, we need to normalize the super-matrix (Saaty,
22
1996). In this example, we assume that the normalizing weights are (.5, .5), meaning that the
growth measures are equally affected by the growth measures themselves and the
introduction measures. We next raise the above matrix to the 25th power:
.4286 .4286 .4286 .4286 .4286 .4286 .4286 .4286 0 .2571 .3750 .3750 .3750 .3750 .3750 .3750 .3750 .3750 0 .2250 .1964 .1964 .1964 .1964 .1964 .1964 .1964 .1964 0 .1179 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1.0 .4 0
0 0 0 0 0 0 0 0 0
A25
The results show that ZGM, the respective weights of ROCE, customer satisfaction, cycle time
reduction, and employee satisfaction, is given by (.4, .1179, .225, .2571). It is interesting to
note that although customer satisfaction is initially rated as the most important future
perspective, it becomes the least important after all of the interactive effects have been
considered. This is because the other two perspectives have a greater impact on the outcome,
albeit indirectly. Cycle time reduction and employee satisfaction are relatively more
important drivers that indirectly contribute to customer satisfaction and subsequent wealth
creation. The company may now implement the BSC as is shown in Table 2.
Table 2. BSC Implementation
Performance Bonus Incentive Actual Incentive
Measure Budget Schedule Performance Earned
(% of Total Bonus) (% of Bonus Budget) (% of Total Bonus)
ROCE 40% 20%+ .................. 100% 14% 13.3%
16%-20% .......... 66.6%
12%-16% .......... 33.3%
Below 12% ............. 0%
Customer 12% Excellent ............ 100% Good 8%
Satisfaction Good ................. 66.6%
Satisfactory ....... 33.3%
Poor ........................ 0%
23
Cycle Time 23% 9%+ .................... 100% 11% 23%
Reduction 6%-9% .............. 66.6%
3%-6% .............. 33.3%
0%-3% ................... 0%
Employee 25% Excellent ............ 100% Excellent 25%
Satisfaction Good ................. 66.6%
Satisfactory ....... 33.3%
Poor ........................ 0%
Total 100% 69.3%
For the assignment of the bonuses to the respective ratings – that is, the assignment of
the percentage of Incentive Schedule (percentage of Bonus Budget) in relation to the
percentage of Bonus (Target Incentive), we assume a linear incremental bonus percentage for
simplicity. In this example, the strong performances in the future measures mean that the
manager will receive a sizable bonus (69.3%), despite having only gained a satisfactory
rating for ROCE.
6. SUMMARY AND CONCLUSION
The Balanced Scorecard (BSC) is an important multi-attribute evaluation framework
that has been accepted by many companies. The BSC advocates the incorporation of non-
financial attributes into the company reward system, and highlights the contribution of non-
financial attributes to the wealth creation process. However, there is a lack of literature that
addresses the actual implementation of the scorecard, and especially the issues of the
determination of the attribute weights, the use of subjective measures, and the proper
handling of time dependencies among the attributes. Because of the inherently inter-related
nature of the attributes, the weight determination process can be quite complex. In this paper,
we have formulated a range of BSC problems with varying levels of complexity, from using a
simple AHP to using a time-dynamic ANP with innerdependencies. We show that inter-
related behaviour between attributes across time can be explicitly incorporated. We also
24
show that subjective criteria can be included. It is believed that the AHP and the ANP are
versatile multi-attribute decision methodologies that can be adapted to a wide range of BSC
decision environments.
In this study, the BSC is examined within the context of performance evaluation.
From a broader perspective, the BSC has been viewed as a vehicle to articulate the strategies
of a company, to communicate these strategies to employees, and to help align individual and
organisational initiatives for the realisation of company goals. In this way, the BSC may be
used as part of a larger management system of communication, information sharing, and
learning (Simon, 1999). Proceeding along this direction, one may use the AHP and the ANP
to study the design of a BSC as a strategic management system. However, we believe the
handling of dependencies will remain a critical and difficult research issue. Future efforts
should be directed towards designing empirical studies (e.g. case analyses and behaviour
experiments) that investigate the effects of dependencies across perspectives and over time.
Only with a clearer understanding of the dependency issue would decision makers be able to
design and implement the BSC as an effective organizational management system.
A recent survey indicates that 70% of the organizations use software packages in
implementing BSC (Lawson et al, 2003). Nowadays, there are a number of software
packages developed for AHP and ANP. They are typically rather user friendly and strive to
provide a mechanism to help users keep track of the complex decision-making process. They
allow inconsistency in the judgments and commonly include ways for improving consistency.
However, these software packages have yet to incorporate the issue of compatibility between
interdependent matrices in ANP (Leung et al., 2003). Also, a major shortcoming of using
AHP and ANP is that it is difficult to validate the model. Indeed, there are quite a variety of
ways to formulate a BSC, such as those suggested in this paper. Whether a particular
formulation is adequate to serve a specific decision making situation needs to be thoroughly
25
tested. Comparing the results of a particular formulation with well-established results is one
way to assess the reliability of an AHP or an ANP model. It is also prudent to compare
results from different formulations. Continual refinement and experimenting with real-life
data are effective ways to safeguard the reliability of using AHP and ANP in BSC.
Acknowledgement: We would like to thank the referees for their thoughtful comments.
References
1. Baiman S and Rajan MV (1995). The informational disadvantages of discretionary
bonus schemes. Acc Rev 70: 557-579.
2. Banker R and Datar S (1989). Sensitivity, precision and linear aggregation of signals
for performance evaluation. J Acc Res 27: 21-39.
3. Belton V and Gear T (1983). On a short-coming in Saaty’s method of analytic
hierarchies. OMEGA 11: 228-230.
4. Brancato CK (1995). New performance measures – a research report. Report Number
1118-95-RR. New York, NY: Conference Board.
5. Chan YL and Lynn BE (1991). Performance evaluation and AHP. J Mngt Acc Res 3:
57-87.
6. Dyson RG (2000). Strategy, performance and operational research. J Opl Res Soc 51:
5-11.
7. Eccles RG and Pyburn PJ (1992). Creating a comprehensive system to measure
performance. Mngt Acc 74: 41-44.
8. Harker PT and Vargas LG (1990). Remarks on the analytic hierarchy process – Reply.
Mngt Sci 36(3): 269-273.
9. Ittner C and Larcker DF (1998). Innovations in performance measurement: trends and
research implications. J Mngt Acc Res 10: 205-238.
10. Kaplan RS and Atkinson AA (1998). Advanced Management Accounting. Third
Edition. Prentice-Hall, NJ.
11. Kaplan RS and Norton DP (1992). The Balanced Scorecard – measures that drive
performance. Harvard Bus Rev 70: 71-79.
26
12. Kaplan RS and Norton DP (1996). The Balanced Scorecard. Harvard Business Press.
Boston. MA.
13. Lawson R, Stratton W and Hatch T (2003). Winning ways: Implementation strategies
balanced scorecard success. CMA Mngt. 77:26-28.
14. Leung LC and Cao D (2001). On the efficacy of modeling multi-attribute decision
problems using AHP and Sinarchy. Eur J Opl Res 132: 39-49.
15. Leung LC, Hui YV, and Zheng M (2003). Analysis of compatibility between
interdependent matrices in ANP. J Opl Res Soc. 54: 758-768.
16. Liberatore MJ and Miller T (1998). A framework for integrating Activity-based
Costing and the Balanced Scorecard into the logistics strategy development and
monitoring process. J Bus Logistics 19: 131-154.
17. Maskell BH (1992). Performance measurement for world class manufacturing.
Corporate Controller 4: 44-49.
18. Rangone A (1996). An analytical hierarchy process framework for comparing the
overall performance of manufacturing departments. Int J Ops Prod Mngt 16: 104-119.
19. Saaty TL (1980). The Analytic Hierarchy Process. New York: McGraw-Hill.
20. Saaty TL (1996). The Analytic Network Process, RWS Publications, Pittsburgh.
21. Simons R (1999). Performance Measurement & Control Systems for Implementing
Strategy. Prentice Hall. New Jersey.
22. Smith PC and Goddard M (2002). Performance management and operational
research: a marriage made in heaven? J Opl Res Soc 53: 247-255.
23. Suwignjo P, Bititci US and Carrie AS (2000). Quantitative models for performance
measurement system. Int J Prod Econ 64: 231-241.
Appendix
Detailed List (Source: Kaplan and Atkinson: Advanced Management Accounting: Chapters 7 to 12).
1. Financial Perspective
Sales: annual growth in sales and profits
Cost of sales: extent that it remains flat or decreases each year
Profitability: EVA or return on total capital employed
Prosperity: cash flows
Company growth versus industry growth
Ratio of international sales to total sales
New product: gross profit/growth from new products
Industry leadership: market share
27
2. Customer perspective
Market share for target customer segment
Customer retention/percentage of growth with existing customers
Customer acquisition: number of new customers/total sales to new customers/actual new
customers divided by prospective inquiries
Customer satisfaction (via satisfaction surveys)
Customer profitability (via accounting analyses)
Customer lead time (on-time delivery)
Quality: parts-per-million (PPM) defect rates, reworks, percentage of returns
3. Internal Business Perspective
Manufacturing cycle effectiveness = processing time/throughput time
Cost of quality comparison
Low cost producer: unit cost versus competitors’ unit cost
Reduce inventory: inventory as percentage of sales
Output per hour/plant utilisation
Safety incident index
4. Learning and growth
Innovation measures
Breakeven time: the time from the beginning of product development work until the product
has been introduced
Rate of new product introduction per quarter
Number of new products with patented technology/new patents
Annual increase in number of new products per engineer
Employee capabilities
Employee satisfaction survey
Employee retention: percentage of key staff turnover
Employee productivity: revenue per employee
Salaries compared to the norm in the local area
Percentage of competency deployment matrix filled
Number of promotions from within
Absenteeism rate
