Content uploaded by Dalila B.M.M. Fontes
Author content
All content in this area was uploaded by Dalila B.M.M. Fontes on Oct 23, 2014
Content may be subject to copyright.
Multicriteria Decision Making: A Case Study in the Automobile Industry
Márcia Oliveira
*a
, Dalila B. M. M. Fontes
a
and Teresa Pereira
b
a
FEP, Faculdade de Economia da Universidade do Porto, and LIAAD/INESC TEC, Rua Dr. Roberto Frias, 4200-
464 Porto, Portugal
b
IPP/ESEIG, Escola Superior de Estudos Industriais e de Gestão, Instituto Politécnico do Porto, CIEFGEI, Rua D.
Sancho I, 981, 4480-876, Vila do Conde, Portugal and Algoritmi Center, Universidade do Minho, 4800-058
Guimarães, Portugal
Abstract
Multi-criteria decision analysis (MCDA) has been one of the fastest-growing areas of operations
research during the last decade. The academic attention devoted to MCDA motivated the development
of a great variety of approaches and methods within the field. These methods distinguish themselves
in terms of procedures, theoretical assumptions and type of decision addressed. This diversity poses
challenges to the process of selecting the most suited method for a specific real-world decision
problem. In this paper, we present a case study for a real-world decision problem arising in the
painting sector of an automobile plant. We tackle the problem by resorting to the well-known AHP
method and to the MCDA method proposed by Pereira and Sameiro de Carvalho (2005) (MMASSI).
By relying on two MCDA methods rather than one, we expect to improve the robustness of the
obtained results. The contributions of this paper are twofold: first, we intend to investigate the
contrasts and similarities of the results obtained by distinct MCDA approaches (AHP and MMASSI);
secondly, we expect to enrich the literature of the field with a real-world MCDA case study on a
complex decision making problem since there is a paucity of applied research work addressing real
decision problems faced by organizations.
Keywords:
AHP, decision making, multicriteria decision analysis, multicriteria methodology,
automobile industry.
1. Introduction
In recent years the increasing competitiveness of the global market, as well as the burst of the so-
called Global Financial Crisis forced companies to rethink their processes in order to raise the
levels of efficiency, responsiveness and flexibility. In such contexts, resorting to MCDA to assist
in strategic decision problems can turn out to be a decisive step towards achieving these goals.
MCDA is a formal quantitative approach to aid the decision making process by fostering in
decision makers (DM) the development of a structured thinking of the decision problem at hand.
The main motivation behind the development of this research field is strongly related to the
recognition that human judgments can be limited, distorted and prone to bias, especially when
faced with problems that require the processing and analysis of large amounts of complex
information (Dodgson et al., 2000). Being aware of such hindrances, in the 60's researchers
started to devote themselves to the development of MCDA methods and techniques in an attempt
to overcome the limitations posed by human judgment. Due to its relevance, MCDA quickly
evolved and established itself as an active research field in the 70's. The proposed methods
*
Corresponding author. E-mail: mdbo@inescporto.pt
2
sought to make the decision making process more structured, transparent and efficient. Besides,
the application of MCDA in real-world problems helps increase the confidence of the decision
makers in their decisions, by helping them to reach a solution that complies with their
preferences and system of values. Due to the interactive and iterative nature of MCDA process,
the application of such techniques to real-world scenarios may prove to be a daunting and time
consuming task, which requires a significant endeavor from both analysts (or facilitators) and
decision makers. Therefore, MCDA is more suitable for supporting problems of high complexity
and that may possibly lead to long term impacts (Brito et al., 2010). In this paper, we adopt the
definitions of decision makers and analysts proposed by Belton & Stewart (2002), thus regarding
the decision maker as the one who has the responsibility for the decision, and the analysts as
those who guide and aid the decision makers in the process of reaching a satisfactory decision.
MCDA is a problem solving methodology that organizes and synthesizes the information
regarding a given decision problem in a way that provides the decision maker with a coherent
overall view of the problem. MCDA methods assist DM in the process of identifying the most
preferred action(s), from a set of possible alternative actions (explicitly or implicitly defined),
when there are multiple, complex, incommensurable and often conflicting objectives (e.g.
maximize quality and minimize costs), measured in terms of different evaluation criteria. The
alternative actions distinguish themselves by the extent to which they achieve the objectives,
since usually none of the alternatives has the best performance for all objectives (Dodgson et al.,
2000). Depending on the typology of the MCDA problem at hand, the alternatives can be
implicitly found by solving a mathematical model or they can instead be explicitly known (Lu et
al., 2007). Criteria (also referred to as attributes or objectives) are performance measures
(qualitative or quantitative) that are ranked by the DM, in terms of their perceived importance,
and considered together when appraising the alternatives. By explicitly assessing the
performance of different alternative actions, based on the integration of objective measurement
with subjective value judgment, MCDA techniques unavoidably lead to more efficient and more
informed decisions. The goal of MCDA is not to prescribe the "best" decision to be chosen but to
help decision makers select a single alternative, or a short-list of good alternatives, that best fit
their needs and is coherent with their preferences and general understanding of the problem
(Brito et al., 2010). Usually, the chosen alternative corresponds to the best compromise solution
rather than to an optimal solution.
The views of academics, such as Belton & Stewart (2002), Seydel (2006) and Dooley et al.
(2009), agree that MCDA prompts learning and better understanding of the perspectives of the
DM themselves and the perspectives of the remaining key players involved in the decision
process. Learning and understanding of the problem is mostly achieved by stimulating reflection,
sharing of ideas, and discussion about the problem at hand. This unavoidably leads to an
increased transparency of the decision making process and might hasten the reaching of
consensus. Thus, MCDA can act as a method to document, support, and justify decisions.
Both the academic attention devoted to the field of MCDA and the application of its
methods in real-world decision problems, are a reflection of the advantages of MCDA
approaches in aiding decision making. Bearing this in mind, in this work we aim at presenting a
case study on a real-world decision problem arising in the painting sector of one of Toyota's
3
plants, using the well-known AHP method (Saaty, 1990) and the MCDA method proposed by
Pereira & Sameiro de Carvalho (2005). The contributions of this paper are twofold: firstly, we
intend to investigate the contrasts and similarities of the results obtained by distinct MCDA
approaches (AHP and MMASSI); secondly, we expect to enrich the literature of the field with a
real-world MCDA application to a complex decision-making problem since there is a paucity of
applied research work addressing real decision problems faced by organizations (Dooley et al.
2009). On a different level, it is also our purpose to encourage the adoption of a more structured
thinking in problem solving by the decision makers of this specific company, since previously
their decisions have been mostly made based on intuition and business experience. By
embedding the principles of the scientific method into the decision-making process, decision
makers are able to work through the problem in a more structured way, improving the objectivity
and transparency of the decision process, as well as their commitment to the decision.
Nonetheless, it should be noted that the main motivation behind this study was not to interfere in
the policies and practices of the company, as it would be if an action research scheme was
adopted but, in turn, to stimulate reflection and present new ways to tackle decision problems.
The remainder of this document is organized as follows. Section 2 describes the steps
involved in the deployment process of MCDA. Section 3 describes the MCDA methods used in
this work, namely, AHP and MMASSI. In Section 4, we provide a detailed description of the
application of these methods to a real-world decision-making problem in the painting sector of
one of Toyota’s plants. Section 5 summarizes the paper and offers some concluding remarks.
2. MCDA Process
The deployment of MCDA is a non-linear recursive process comprising several stages. The
number of stages varies according to the adopted MCDA approach, since each one has its own
idiosyncrasies. Nevertheless, it is possible to outline the critical steps of a generic MCDA
process that traverse the great majority of MCDA approaches.
Usually, the first step towards the application of MCDA in real-world problems is related
to both the establishment of a common understanding of the decision context and the
identification of the decision problem. This step involves the decision makers and other key
players that are able to make significant contributions to the MCDA process through the sharing
of their expertise.
The shared perception of the decision context is acquired by means of the
understanding of the objectives of the decision making body and the identification of not only
the set of people that are responsible for the decision, but also those that are likely to be affected
by the decision (Dodgson et al., 2000). The second and third steps of the process comprise the
identification of both the alternatives and the decision criteria that are relevant for appraising
these possible courses of action. According to Dooley et al. (2009), these initial three steps are
usually the most time-consuming tasks of a MCDA process, especially due to their qualitative
nature. The step that follows is the assignment of relative importance weights to the chosen
criteria. These weights can be determined directly (e.g. rating, ranking, swing, trade-off) or
indirectly (e.g. centrality, regression, and interactive). Afterwards, the DM is asked to allot a
subjective score, reflecting his/her opinions, to each one of the identified alternatives according
4
to the criteria deemed important. These scores reflect the judgment of the DM in terms of the
contribution of each alternative to each performance criterion. The information thus obtained is
typically organized into the so-called performance matrix (also referred to as consequence
matrix, options matrix, or simply decision table), where the rows and columns correspond to the
alternatives and the criteria, respectively, and each entry represents the performance of each
alternative against each criterion. The following step of the process involves the summarization
of the information comprised in the performance matrix into a set of multi-criteria scores, one for
each possible course of action. Usually, this is achieved by aggregating (implicitly or explicitly)
the subjective scores of the matrix so as to derive an overall assessment for each alternative that
allows further comparison. Based on these overall scores the set of alternatives is ranked.
Eventually, the process may also involve a sensitivity analysis of the results to changes in scores
or criteria, in order to infer on the robustness of the outcome of MCDA. Finally, the evaluation
and trade-offs involved on the considered alternatives are provided to and discussed with the
DM. In most cases, the final decision taken by the DM does not correspond to the top-ranked
alternative, since they tend to be more concerned with the process of understanding the impact of
each criterion in the ranking of alternatives than in the accuracy of the ranking (Dooley et al.,
2009). Moreover, it is important to note that the results yielded by a MCDA process are not
prone to generalizations, in the sense that they only apply to the set of alternatives that were
evaluated.
3. MCDA Methods
Although several methods have been proposed over the years, here we only describe the AHP
and MMASSI, since these are the ones used in our study. Before presenting these methods, we
first introduce the main schools of thought in the field.
3.1. Dominant Schools of Thought in MCDA
There are two major schools of thought in MCDA that govern the methods proposed in this field:
the French school, represented by the ELimination and (Et) Choice Translating REality
(ELECTRE) family of outranking methods (Roy, 1991); and the American school represented by
the Analytic Hierarchy Process (AHP), proposed by Saaty in the 80’s (Saaty, 1986, 1990). These
dominant schools share the same goal since both are concerned with the problem of assessing a
finite set of alternatives, based on a finite set of conflicting criteria, by a decision making body.
However, they differ in the way they approach the decision problem. According to Lootsma,
methods arising from the French school "model subjective human judgment via partial systems
of binary outranking relations between the alternatives and via a global system of outranking
relations", while methods from the American school build "partial value functions on the set of
alternatives, as well as a global value function" (Lootsma, 1990, page 282). Analogous
distinctions can be made at lower levels of the taxonomy of MCDA methods since even methods
within the same school distinguish themselves in terms of procedures and theoretical
assumptions. These peculiarities should be borne in mind when selecting the most suited MCDA
5
approach to a specific decision problem, due to the lack of consistency of the obtained results. In
other words, the application of different methods to the same decision problem may yield
different results. Hanne (1999) pointed out three important aspects that should be taken into
account when selecting a MCDA method in a real-world decision context, namely:
characteristics of the problem at hand, the method requirements, and the DM requirements. The
characteristics of the problem are related to the categories in which a given MCDM problem
falls. More specifically, if the problem has a continuous set of alternatives, it can be framed as a
Multi-objective Decision Making (MODM) problem, whilst if the decision space is discrete, the
problem falls within the category of Multi-attribute Decision Making (MADM). The proper
identification of the nature of a given decision problem is of utter importance, since some
MCDA methods are only able to handle one of the mentioned types (e.g. interactive approaches
were devised to solve MODM problems, whereas AHP or outranking approaches, are only able
to deal with MADM). Other problem types can be found both in real life and in the literature.
Examples include problems with discrete, integer, or binary and stochastic or fuzzy decision
variables (van Laarhoven & Pedrycz, 1983).
3.2. Methods
3.2.1. AHP
One of the most prevalent and popular approaches for MCDA is AHP. This problem solving
framework was originally developed by the mathematician Thomas Saaty (1986, 1990), in the
late 70’s. AHP belongs to the family of normative methods of the American school of thought.
Albeit the severe criticism and heated debate that AHP has been subjected to by MCDA
academics, its widespread application reflects its generalized acceptance by both the academic
and practitioner communities.
The basic idea behind AHP is to convert subjective assessments of relative importance into
a set of overall scores and weights. The assessments are subjective since, on the one hand they
reflect the perception of the DM, and on the other hand, they are based on pairwise comparisons
of criteria/alternatives. The first step of AHP is to decompose the decision problem into a
hierarchy of subproblems, by arranging the relevant factors of the problem into a hierarchic
structure that descends from an overall goal to criteria, sub-criteria and alternatives, in successive
levels. According to Saaty (1990), the higher levels of the AHP hierarchy should represent the
elements with global character (e.g. the main objective of the decision problem), while the levels
with greater depth should be devoted to the elements that have a more specific nature (e.g.
multiple criteria and alternatives). Resorting to this type of hierarchies provides the DM with an
overall view of the complex relationships inherent in the decision problem, fostering a better
understanding of the problem itself. The second step of the method comprises the elicitation of
pairwise comparison judgments from the decision making body. Here, the DM is asked to assess
the relative importance of criteria with respect to the overall goal, through pairwise comparisons
(e.g. criterion A with criterion B; criterion A with criterion C). The same procedure can be
employed to appraise the alternatives, according to the degree to which they satisfy each
6
criterion. The output of this preference elicitation process is a set of verbal answers of the DM,
which are subsequently codified into a nine-point intensity scale. This semantic scale was
proposed by Saaty (1986) and assumes discrete values from 1 (equally preferable) to 9 (strongly
preferable), where the values 2, 4, 6, and 8 represent intermediate values of preference.
One of the distinguishing characteristics of AHP is the fact that it is grounded in pairwise
comparisons, which are often regarded as straightforward, intuitive and convenient means to
extract subjective information from the DM. However, pairwise comparison strategies rely on
the assumption that the DM is consistent in his/her judgments, which is not always guaranteed in
practice. To measure the degree to which the DM was consistent in his/her responses, a
consistency index is computed for a given matrix. If its value is higher than a specific value
( ) (Saaty, 1986, 1990), then the matrix entries need to be amended since there were
inconsistencies in the DM judgments.
The questions asked to the DM in the previous step of the AHP process aim at achieving
two goals: derive and estimate the priorities or weights of criteria and establish the relative
performance scores for alternatives in each criterion. After the determination of the pairwise
comparisons among criteria, AHP converts the corresponding DM evaluations into a vector of
priorities, by finding the first eigenvector of the criteria matrix. This vector has information
about the relative priority of each criterion with respect to the global goal. The following step of
AHP, which involves the relative importance of criteria, can be performed using two approaches:
one supported on the relative measurement of alternatives and another based on absolute
measurements of these alternatives (Saaty, 1990). In the former approach, separate pairwise
comparisons for the set of alternatives in each criterion (and sub-criterion, if applicable) are
carried out in order to elicit their performance scores. In the latter approach, the alternatives are
simply rated in each criterion, by identifying the grade that best describes them (Saaty, 1990).
Afterwards, a weighting and summing step yields the final results of AHP, which are the
orderings of the alternatives based on a global indicator of priority. The alternative with the
largest value of this global score is the most preferred one.
The main reasons behind the wide applicability of AHP are: its simplicity, since it does not
involve cumbersome Mathematics; the relative ease with which it handles multiple criteria; its
great flexibility, being able to effectively deal with both qualitative and quantitative data; and the
ease of understanding (Kahraman et al., 2003). Besides, the consistency verification operation of
AHP can act as a feedback mechanism for the DM to review and revise the judgments, thus
preventing inconsistencies (Ho et al., 2009). However, despite these advantages, the drawbacks
of AHP instigated a controversial debate among MCDA academics that raises doubts about the
underlying theoretical foundations of the method. The major concerns are closely related to the
rank reversal problem and to the potential inconsistency of the nine-point scale proposed by
Saaty (1986). Rank reversal occurs whenever the addition of one alternative to the initial set of
alternatives modifies the final relative ordering of the alternatives (Goodwin & Wright, 2004).
This situation may lead to different solutions, even if the relative judgments remain unchanged.
Regarding the nine-point scale, it was identified a lack of theoretical foundation between the
points used in the scale and the corresponding verbal description (Goodwin & Wright, 2004).
The effect of the order of the elicitation process can also be understood as a problem because,
7
since criteria priorities are elicited before the performance scores of alternatives, the DM is
induced to make statements about the relative importance of items without knowing, in fact,
what is being compared (Dodgson et al., 2000). According to Dyer (1990), one of the main flaws
of AHP is the ambiguity of the elicitation questions, since they require that the DM explicitly, or
implicitly, determines a reference point in the ratio scale. Seydel (2006) also mentions that the
large number of comparisons required by AHP, especially when dealing with a large number of
criteria and/or alternatives, can turn the pairwise comparisons into a cumbersome and time-
consuming task.
These issues lead us to use another method so that a more confident evaluation and
analysis can be provided to the DM.
3.2.2. MMASSI
Here we perform a comparison of the results yielded by the well-known AHP method and the
ones provided by MMASSI. This way, we are able to increase the level of confidence on the
yielded results, by removing some of the constraints associated to the use of a single method.
MMASSI was first proposed by Pereira & Sameiro de Carvalho (2005) and further extended to
group decision making by Pereira & Fontes (2012). The underpinnings of MMASSI rely on
existing normative methods, which were developed along the lines of the American school of
thought. This methodology was originally devised to aid the decision support process involving a
group of decision makers. However, to fit the scope of our research, we will adapt it to a single-
decision maker (or a consensual group of them). MMASSI can be distinguished from previously
proposed MCDA methodologies in the sense that (a) it provides the DM with a pre-defined set of
criteria that tries to generally cover all the relevant criteria in the field of application; (b) it does
not explicitly requires the presence of a facilitator, or analyst, to guide the DM throughout the
decision process, since it is implemented in an user-friendly and self-explanatory software; and
(c) it uses a continuous scale with two reference levels and thus no normalization of the
valuations is required.
MMASSI methodology encompasses a set of sequential steps that guide the DM through
the several stages of a multi-criteria decision process. MMASSI begins by presenting the DM
with a pre-defined set of criteria, along with their descriptions and suggestions on how to
measure them. These criteria are chosen based on the a priori study of the decision context and
subsequent identification of the features that are consensually considered relevant within its
scope. This provisional family of criteria works as a starting point to guide the DM through the
criteria selection. Nevertheless, it is the DM who defines and assesses the suggested criteria
according to the following range of properties: completeness, redundancy, mutual independence
and operationality (Seydel, 2006). In order to generate the final set of criteria, the DM can refine
the starting set by removing, modifying, or adding criteria. After validating the criteria set, a set
of alternatives is provided by the DM, or the analyst if one is involved, to the MMASSI system.
The following stage comprises the employment of a weighting elicitation technique, namely the
swing-weight procedure proposed by Winterfeldt & Edwards (1986), which sets up the relative
criteria weights according to the preferences expressed by the DM. A fixed continuous scale with
8
seven semantic levels with two references is presented to the DM so as to set up the ground
values based on which he/she assesses each considered alternative against each selected
criterion. The construction of this scale was based on earlier work by Bana e Costa & Vansnick
(1999). The considered levels are the following: Much Worse, Worse, Slightly Worse, Neutral,
Slightly Better, Better and Much Better. This stage implies a mandatory a priori definition of
two reference scale levels, namely, the "Neutral" (or indifference level) and the "Better" levels,
which are to be used to assess each alternative in each criterion. This interval scale is fully
defined by the DM, taking into account the business and organizational context of the analysis,
and it should mirror his/her preferences. Having defined the criteria, the possible courses of
action and a continuous semantic scale, the DM, in the next phase, appraises each alternative by
allotting a semantic level to each criterion. The chosen level should reflect the subjective
preferences and individual judgments of the DM in terms of the extent to which a given
alternative achieves the objectives. The last step of MMASSI involves the computation of an
overall score for each alternative, according to an additive aggregation model, and the
subsequent ranking of the alternatives. Similar to AHP method, the alternative ranked first is
associated to the largest overall score and corresponds to the most preferred alternative.
MMASSI also offers the possibility of performing a sensitivity analysis to assess the robustness
of the preference ranking to changes in the criteria scores and/or the assigned weights.
Sensitivity analysis measures the impact of small perturbations in the variables of the problem
(e.g. criteria scores and criteria weights) in terms of alternatives, by means of the comparison of
the modified ranking with the original one. The closer the rankings, the more robust the method
is. These steps are important to increase the DM’s confidence in the outcome of the multi-criteria
decision analysis.
4. Case Study: Evaluation of Vehicle Painting Plans
The automobile industry has been one of most affected by the global financial downturn, since it
led to a sharp fall on industry sales. Due to this reason, the automobile plant where we carried
out our case study is producing below capacity. Under such adverse circumstances, the
management of the plant felt the need to optimize its processes. Since the painting process is (a)
one of the most complex activities in automobile manufacturing, (b) a bottleneck in this specific
plant, and (c) responsible for the highest costs (e.g. the painting sector costs represent a fraction
of, approximately, 70% of the total expenditures of the entire plant), the plant manager
considered this sector to be the most critical to conduct a MCDA.
The purpose of this case study is twofold. On the one hand, we illustrate the potential of
the application of MCDA for solving a complex decision making problem in the painting sector
of an automobile plant. On the other hand, we analyze the possible vehicle painting plans in
order to provide the DM with an evaluation of the aforementioned plans, as well as with the
identification of the one that contributes the most to the painting process optimization.
In this section we describe the decision problem under consideration, explain how the case
study was carried out, and present the obtained results, by traversing each one of the stages
identified in Section 2.
9
Figure 1. Illustration of the painting process
4.1. Problem Description
The target of our case study is one of the plants of Toyota, located in Ovar, Portugal. The main
function of this plant is to perform the welding, painting, and final assembly of a specific
automotive model. The vehicle parts are delivered to the plant in batches. Each batch includes the
necessary parts to assemble five vehicles. After selecting the necessary parts for production,
according to the production planning, these parts are forwarded to the welding sector. The
welded bodywork of the vehicles is then directed to the painting sector. Since our work focuses
exclusively in this sector, we will later describe the painting process in detail.
The management is interested in optimizing the painting process, which is a bottleneck of
the plant. The only way to improve this process is by optimizing the vehicle painting plans.
These painting plans are defined as a combination of a vehicle type, which can be simple or
mixed, with the number of distinct colors used to paint the vehicles, in a given day. Given this,
the purpose of this case study is, on the one hand, to illustrate the potential of the application of
MCDA for solving a complex decision making problem in the painting sector of an automobile
plant; on the other hand, to provide the DM with an evaluation of the aforementioned painting
plans, as well as with information about the most preferred plan.
4.1.1. Description of the Painting Process
The painting sector comprises a production line which is made up of a series of work stations.
Figure 1 displays the general job flow of the painting process. When the vehicles bodies (or
simply cabins, in this case) are transferred to the paint shop, they are first subjected to a prewash.
The main process begins in the next station, where the surface of the cabins is cleaned and
prepared for the subsequent application of organic coatings through a chemical pretreatment.
Then, the surfaces of the cabins are washed again and further submitted to electrocoating.
Afterwards, they are dried in an oven, with the purpose of baking the coat of paint, and subjected
to a manual inspection.
If any defect is detected, it is repaired by manual sanding. It follows the
application of sealing and PVC to prevent humidity penetration and protect from corrosion. The
sealing is dried in another oven and then the cabins are wiped. The cabins are subsequently
10
subjected to a primer painting, in a spray booth, and dried in an oven. The goal of the primer
painting is to prepare the surface of the cabins to the top-coat application. The operations
performed at work stations 12, 13, 14 and 15 are repeated when applying the top-coat. The
process continues with the manual inspection of the physical aspect of the painted surface. In
case defects are detected, they are repaired through manual sanding and rectification. The
painting process ends with the application of anti-corrosive wax. The painted cabins are then
stocked in a temporary warehouse until being forwarded to final assembly.
Figure 2. The decision hierarchy of the decision problem at hand. At the top level is the main
goal of the decision making. The second level consists of the criteria that contribute to the
overall goal. The third level is comprised of the alternatives that will be evaluated in terms of the
criteria of the second level.
4.2. Data Gathering
The application of MCDA to this decision problem involved the operations manager of the plant
and the painting sector team (henceforth decision maker, or simply DM). Albeit there are several
people involved in the decision making process, they act as if they were a single decision maker,
since the given answers represent the consensual views and preferences of both the manager and
the painting sector team. A number of face-to-face meetings with the DM were convened, so as
to understand the decision context and gather information regarding the decision problem, the
alternatives, and the relevant criteria.
As mentioned before, the goal of the DM is to optimize the painting sector of the plant
through the optimization of the vehicle painting plans. The portfolio of alternatives was
determined by identifying the most frequent painting plans based on daily historical data of the
plant. The analyzed data referred to a time span of six months (June 2012 to December 2012).
Using this procedure we identified eight alternatives, which will be referred to, in this paper, as
PP-A (Painting Plan A), PP-B, through to PP-H. These alternatives were validated by the DM
and are described in Table 3.
11
Table 1. AHP pairwise comparison matrix for criteria and the corresponding criteria weights.
QI
EC
PC
NPV
Criteria Weights
Quality Index (QI)
Energy Consumption (EC)
Paint Consumption (PC)
Number of Painted Vehicles (NPV)
1
1/3
1/7
1/9
3
1
1/2
1/7
7
2
1
1/5
9
7
5
1
0.606
0.23
0.126
0.039
The next step was the selection of the relevant set of criteria to be used to appraise each
one the alternatives. Four quantitative criteria were considered after a brainstorming session with
the DM, namely: the quality index, the energy consumption, the paint consumption and the
number of painted vehicles. The quality index (QI) is given by the average number of defects per
painted vehicle and, as the name implies, it is a proxy for the quality of the performed painting.
Defects can arise as a result of the manual painting process, which is performed by painters, or as
a consequence of the ink quality. Energy consumption (EC) includes both the electricity and the
gas consumption of the painting sector and is measured in kilowatts-hour (kWh). Note that, for
the purpose of this research and for the sake of coherency, gas consumption was converted to
kWh. In turn, paint consumption (PC) reflects the direct cost of painting the vehicles (in terms of
materials), being given by the average ink liters used to paint a given vehicle. The last criterion is
the number of painted vehicles (NPV) per day. More detailed information regarding these criteria
is given in Table 3.
Based on the gathered information, the decision problem is unbundled into its constituent
parts using a AHP hierarchy tree structure comprising three levels (overall goal, criteria, and
alternative painting plans), as depicted in Figure 2. This hierarchical tree has the advantage of
providing an overall view of the complex relationships inherent in the decision problem, thus
easing the understanding of the problem by the DM.
4.3. Elicitation of Criteria Weights
After structuring the decision problem at hand, the DM was asked to assess the relative
importance of the identified criteria based on two different procedures: pairwise comparisons and
swing-weight procedure of Winterfeldt & Edwards (1986). The former is used in the AHP
method, whereas the latter is used in the MMASSI methodology.
These weights are non-negative numbers and independent of the measurement units of the
criteria, and are determined such that higher weights reflect higher importance. The sum of the
normalized weights equals 1, which implies that each criterion can be interpreted according to
their proportional importance.
12
Table 2. Swing-weight scores, as given by the DM, and the corresponding normalized criteria
weights obtained by MMASSI. Criteria Swing
Weights
Quality Index (QI)
Energy Consumption (EC)
Paint Consumption (PC)
Number of Painted Vehicles (NPV)
100
40
20
10
0.588
0.235
0.118
0.059
Total 170 1
4.3.1. AHP
According to AHP, the assignment of weights to the chosen criteria is performed by asking the
DM to form an individual pairwise comparison matrix using the nine-point intensity scale
proposed by Saaty (1990). In this pairwise comparison matrix, the four criteria are compared
against each other in terms of their relative importance, or contribution, to the main goal of the
decision problem. Table 1 shows the pairwise comparison judgments provided by the DM, as
well as the resulting criteria weights. Based on the AHP results, the quality index was deemed
the most important criterion ( ) for the evaluation of the painting plans, followed by
energy consumption ( ) and paint consumption ( ). The least important
criterion is the number of painted vehicles, which was assigned a relative importance of merely
3.9%.
A pairwise comparison matrix is of acceptable consistency if the corresponding
Consistency Ratio (CR) is (Saaty, 1990). Since we obtained , the DM
has been consistent in his judgments and, thus, the obtained criteria weights can be used in the
decision making process.
4.3.2. MMASSI
Contrary to AHP, which relies on pairwise comparisons, MASSI resorts to the swing-weight
procedure to derive criterion weights. According to this procedure, the DM should first identify
the most important criterion, to which a score of 100 is assigned, and then successively allot
relative scores (lower than 100) to the second, third and fourth most important criteria. The given
scores should reflect the DM's order and magnitude of preference and are further normalized so
as to obtain the criteria weights. Table 2 provides both the DM’s scores and the resulting criteria
weights. The comparison of Table 2 with Table 1 shows a considerable similarity between the set
of criteria weights obtained by AHP and the ones returned by MMASSI. This similarity indicates
consistency in the DM’s judgments. Once again, quality index is the criterion with highest
priority, with an influence of 58.8%, followed by the energy consumption ( ), paint
consumption ( ), and finally number of painted vehicles ( ).
13
Table 3. Performance Matrix. The best values observed for each criterion are underlined.
Criteria QI EC PC NPV
Unit
Max/Min # Defects
Min kWh
Min Ink liters
Min # Vehicles
Max
Weights AHP
Weights MASSI 0.606
0.588 0.23
0.235 0.126
0.118 0.039
0.059
PP
-
A
(Simple + 1 Color)
PP-B (Simple + 2 Colors)
PP-C (Simple + 3 Colors)
PP-D (Simple + 4 Colors)
PP-E (Mixed + 1 Color)
PP-F (Mixed + 2 Colors)
PP-G (Mixed + 3 Colors)
PP-H (Mixed + 4 Colors)
3.45
2.1
1.6
3.2
2.1
3.0
2.8
2.5
87
66
60
79
81
73
72
53
2.02
1.85
1.59
1.87
1.55
1.58
1.64
1.56
15
14
30
15
11
21
16
15
4.4. Evaluation and ranking of the alternatives
In this stage, the alternative painting plans are appraised by the DM in terms of their contribution
to the previously stated criteria. To obtain this information, we have asked the DM to provide a
numerical evaluation of the relative performance of each alternative painting plan for each
considered criterion. These numerical evaluations are expressed using the scale adopted by each
MCDA approach (e.g. AHP uses the nine-point intensity scale). To assist the DM in this stage,
we constructed a performance matrix by aggregating the daily data gathered by the painting
sector, for a period of six months (June 2012 to December 2012). This matrix provides objective
information regarding the performance of each alternative on each relevant criterion, and served
as a basis for the DM's evaluation. Upon completion of this stage, the overall score of each
alternative is computed based on an aggregation procedure that takes into account, not only the
alternatives performance evaluation provided by the DM, but also the criteria weights. The final
ranking is generated by sorting the alternatives in decreasing order of the overall scores.
4.4.1. AHP
In this step, the DM is asked to appraise the alternatives by performing separate pairwise
comparisons for the set of alternatives in each criterion. This elicitation process is based on a set
of questions of the general form: “How much more does alternative 1 contributes to the
achievement of criterion A than alternative 2?”. The corresponding verbal answers of the DM are
written down and subsequently codified into the nine-point intensity scale of AHP. These
relative performance scores constitute one of the inputs of a weighting and summing step that
yields the final result of AHP.
14
Table 4. Mandatory reference scale levels of MMASSI, as defined by the DM, for each criterion.
Reference Scale Levels Neutral
Better
Quality Index
Energy Consumption
Paint Consumption
Number of Painted Vehicles
1.8
27
1.79
12
1.6
21
1.66
21
Table 5. Final rankings yielded by AHP and MMASSI methods. The overall scores range from 0
to 100, with a higher score representing a higher level of preference.
AHP Ranking AHP Overall Score MMASSI Ranking MMASSI Overall Score
PP-C 88.29
PP-H 49.53
PP-E 49.08
PP-B 46.92
PP-F 21.9
PP-G 21.71
PP-D 9.09
PP-A 5.16
PP-C 64.64
PP-E 51.05
PP-B 33.69
PP-H 28.09
PP-G 22.95
PP-F 21.72
PP-D 12.96
PP-A 8.72
4.4.2. MMASSI
Regarding MMASSI, the DM was first asked to set, for each criterion, the mandatory reference
levels (neutral and better levels) of MMASSI fixed scale (c.f. Section 3.2.2). These levels are
expressed in the original units of measurement of criteria. The reflection instigated by the need to
define these levels prompted the DM to review and adjust the painting sector goals for each
criterion. The established levels are shown in Table 4. Taking into account these two reference
levels, the DM appraises the set of alternatives, on each criterion, by assigning one of the
following semantic levels to each alternative: Much Worse, Worse, Slightly Worse, Neutral,
Slightly Better, Better or Much Better. In this MCDA step, the major differences between AHP
and MMASSI are the following: (a) in contrast with AHP, MMASSI does not rely on pairwise
comparisons, since each alternative is only assessed in terms of its contribution to each criterion;
(b) instead of using the potentially inconsistent nine-point semantic scale of AHP, MMASSI
relies on a fixed interval scale that is fully defined by the DM.
4.4.3. Comparison of Results
After performing these evaluations, the alternatives were ranked based on a global indicator of
preference. From the analysis of Table 5, we deduce that the most preferred alternative is PP-C,
since it ranks first in both AHP and MMASSI final rankings. Thus, the panting plan with highest
relative merit is the one involving the painting of simple vehicles with three different colors.
From the business viewpoint, this result means that PP-C is the painting plan which contributes
the most to the painting process optimization. In order to compare the similarity of the rankings
returned by the two methods, we compute Kendall's tau rank correlation coefficient (Kendall,
15
1938), denoted as ( ). The obtained value, , indicates the existence of a
significant rank correlation between the AHP and MMASSI final rankings, which means that
both methods yield quite similar results.
4.5. Sensitivity Analysis
Since some steps of the MCDA process can be permeated by subjectivity and uncertainty, we
validated our results by performing a sensitivity analysis in order to determine how the final
ranking of alternatives changes under different criteria weighting schemes. The results for both
AHP and MMASSI have shown that changes in the relative criteria weights did not make any
impact on both the top (i.e. first and second positions) and the bottom (i.e. seventh and eight
positions) of the ranking, although some position shifts were observed in the intermediate
ranking levels (namely, in the third and sixth positions). These conclusions also hold when
introducing considerable changes on the criteria weights, and also for the case in which criteria
have equal priorities.
5. Conclusions
In this paper, we report the application of MCDA to a case study on the automobile industry. The
goal of this case study is to assist the management of the automobile plant in the process of
evaluating the relative merits of alternative painting plans, so as to optimize the painting sector.
This problem is of great relevance for the company, since the painting sector is a bottleneck of
the manufacturing process of the plant. Being aware that MCDA methods are prone to
subjectivity and uncertainty, we resorted to two MCDA methods, namely the well-known AHP
and MMASSI, the MCDA method proposed by Pereira & Sameiro de Carvalho (2005), in order
to increase the confidence, reliability, and robustness of the obtained results.
According to DM's point of view, MMASSI method proved to be swifter and easier to
understand during the preference elicitation stage. This is partly explained by the use of a
continuous scale, rather than semantic one, and by the requirement of a lower number of
evaluations, when compared to AHP. Nevertheless, AHP proved to be more advantageous than
MMASSI for structuring the decision problem. The application of the MCDA methodology
encouraged fruitful discussions and a deeper analysis of the problem peculiarities among the
team. This reflection, along with the process of gathering and summarizing the historical data of
the plant, helped the team to determine the right key performance indicators and the
corresponding target values for the painting sector. Other goals were also achieved, namely, by
providing the team with a framework to address and solve complex problems in a more
structured and scientific way. Regarding the MCDA results, the management found the results
valuable and intends to use the final rankings to enhance the weekly planning of the painting
sector.
We acknowledge that this study is limited in ways that suggest opportunities for future
research. A possible direction would be to solve this decision problem using integrated
16
approaches that combine the strengths of different MCDA methods. We also intend to explore
more formally the distinguishing properties of MMASSI in relation to AHP.
Acknowledgments
The funding by the ERDF (FEDER) through the COMPETE Programme and by FCT within the
project PTDC/EGEGES/099741/2008 is acknowledged. First author was also funded by FCT,
under the PhD grant SFRH/BD/81339/2011. The authors are also thankful to Toyota for
providing the research scenario and the necessary conditions to carry out this case study.
References
Bana e Costa, A. C., & Vansnick, J.-C. (1999). The MacBeth approach: Basic idea, software and
an application. In Meskens, N. and Roubens, M. (Eds.), Advances in decision analysis (pp.
131-157). Dordrecht, The Netherlands: Kluwer Academic Publishers.
Belton, V., & Stewart, T. J. (2002). Multiple Criteria Decision Analysis: An Integrated
Approach. Dordrecht, The Netherlands: Kluwer Academic Publishers.
Brito, T. B., Silva, R. C. d. S., Botter, R. C., Pereira, N. N., & Medina, A. C. (2010). Discrete
event simulation combined with multi-criteria decision analysis applied to steel plant
logistics system planning. In Winter Simulation Conference, pages 2126–2137, Baltimore,
MD, USA.
Dyer, J. S. (1990). Remarks on the Analytic Hierarchy Process. Management Science, 36(3),
249-258.
Dodgson, J., Spackman, M., Pearman, A., & Phillips, L. (2000). Multi-criteria analysis: a
manual. Technical report, Department of the Environment Transport and the Regions.
Dooley, A. E., Smeaton, D. C., Sheath, G. W., & Ledgard, S. F. (2009). Application of multiple
criteria decision analysis in the New Zealand agricultural industry. Journal of Multi-
Criteria Decision Analysis, 16(1-2), 39-53.
Goodwin, P., & Wright, G. (2004). Decision Analysis for Management Judgment. (3rd ed.),
Chichester, United Kingdom: John Wiley and Sons.
Hanne, T. (1999). Meta decision problems in multiple criteria decision making. In Gal, T.,
Stewart, T. J., & Hanne, T. (Eds.), Multicriteria Decision Making: Advances in MCDM
Models, Algorithms, Theory and Applications (pp. 6.1-6.25). Boston, USA: Springer US.
Ho, W., Xu, X., & Dey, P. K. (2009). Multi-criteria decision making approaches for supplier
evaluation and selection: A literature review. European Journal of Operational Research,
202(1), 16-24.
Kahraman, C., Cebeci, U., & Ulukan, Z. (2003). Multi-criteria supplier selection using fuzzy
AHP. Logistics Information Management, 16(6), 382-394.
Kendall, M. G. (1938). A new measure of rank correlation. Biometrika, 30(1/2), 81-93.
Lootsma, F. A. (1990). The French and the American school in multi-criteria decision analysis.
Recherche Opérationelle, 24(3), 263-285.
17
Lu, J., Zhang, G., Ruan, D., & Wu, F. (2007). Multi-objective Group Decision Making: Methods,
Software and Applications with Fuzzy Set Techniques. London, UK: Imperial College
Press.
Pereira, T., & Sameiro de Carvalho, M. (2005). A Decision Support System for Selection of
Information Systems and Information Technologies in distribution Companies. In Kulwant
S. Pawar, Chandra S. Lalwani, José Crespo de Carvalho and Moreno Muffatto (Eds.),
Proceedings of the 10th International Symposium On Logistics: Innovations in Global
Supply Chain Networks. Lisbon, Portugal.
Pereira, T., & Fontes, D. B. M. M. (2012). Group decision making for selection of an
information system in a business context. In DA2PL’2012 Workshop: From Multiple
Criteria Decision Aid to Preference Learning (pp. 74-82). Mons, Belgium.
Roy, B. (1991). The outranking approach and the foundations of ELECTRE methods. Theory
and Decision, 31(1), 49-73.
Hanne, T. (1999). Meta decision problems in multiple criteria decision making. In Gal, T.,
Stewart, T. J., & Hanne, T. (Eds.), Multicriteria Decision Making: Advances in MCDM
Models, Algorithms, Theory and Applications (pp. 6.1-6.25). Boston, USA: Springer US.
Roy, B. (1999). Decision-Aiding Today: What Should We Expect? In Gal, T., Stewart, T. J., &
Hanne, T. (Eds.), Multicriteria Decision Making: Advances in MCDM Models, Algorithms,
Theory and Applications (pp. 1.1-1.35). Boston, USA: Springer US.
Saaty, T. L. (1986). Axiomatic foundation of the Analytic Hierarchy Process. Management
Science, 32(7), 841-855.
Saaty, T. L. (1989). Group Decision Making and the AHP. In Golden, B. L., Wasil, E. A., and
Harker, P. T. (Eds.), The Analytic Hierarchy Process: Applications and Studies (pp. 59-
67). New York, USA: Springer
Berlin Heidelberg.
Saaty, T. L. (1990). How to make a decision: The Analytic Hierarchy Process. European Journal
of Operational Research, 48(1), 9-26.
Seydel, J. (2006). Data Envelopment Analysis for Decision Support. Industrial Management and
Data Systems, 106(1), 81-95.
van Laarhoven, P. J. M. & Pedrycz, W. (1983). A fuzzy extension of Saaty’s Priority Theory.
Fuzzy Sets and Systems, 11(1-3), 199-227.
Winterfeldt, D. V., & Edwards, W. (1986). Decision Analysis and Behavioral Research.
Cambridge, New York, USA: Cambridge University Press.
