ArticlePDF Available

Abstract and Figures

Multicriteria decision analysis (MCDA) has been one of the fastest-growing areas of operations research during the last decade. The research attention devoted to MCDA motivated the development of a great variety of approaches and methods within the field. These methods differentiate themselves in terms of procedures, theoretical assumptions and type of decision addressed. This diverseness of these methods poses a great challenge to the process of selecting the most suitable method for a specific real-world decision problem. In this paper, we present a case study for a real-world decision problem in the painting department of an automobile assembly plant. We solved the problem by applying the well-known AHP method and the MCDA method proposed by Pereira and Sameiro de Carvalho (2005) (MMASSI). By applying two MCDA methods rather than one, we expect to improve the robustness of the results obtained. The contributions of this paper are twofold: first, we intend to compare the results obtained with the two MCDA methods (i.e. AHP and MMASSI). Secondly, we intend to enrich the literature in the field with a real-world MCDA case study on a complex decision making problem, since there is a paucity of research work addressing real-world decision problems faced by organizations.
Content may be subject to copyright.
Multicriteria Decision Making: A Case Study in the Automobile Industry
Márcia Oliveira
*a
, Dalila B. M. M. Fontes
a
and Teresa Pereira
b
a
FEP, Faculdade de Economia da Universidade do Porto, and LIAAD/INESC TEC, Rua Dr. Roberto Frias, 4200-
464 Porto, Portugal
b
IPP/ESEIG, Escola Superior de Estudos Industriais e de Gestão, Instituto Politécnico do Porto, CIEFGEI, Rua D.
Sancho I, 981, 4480-876, Vila do Conde, Portugal and Algoritmi Center, Universidade do Minho, 4800-058
Guimarães, Portugal
Abstract
Multi-criteria decision analysis (MCDA) has been one of the fastest-growing areas of operations
research during the last decade. The academic attention devoted to MCDA motivated the development
of a great variety of approaches and methods within the field. These methods distinguish themselves
in terms of procedures, theoretical assumptions and type of decision addressed. This diversity poses
challenges to the process of selecting the most suited method for a specific real-world decision
problem. In this paper, we present a case study for a real-world decision problem arising in the
painting sector of an automobile plant. We tackle the problem by resorting to the well-known AHP
method and to the MCDA method proposed by Pereira and Sameiro de Carvalho (2005) (MMASSI).
By relying on two MCDA methods rather than one, we expect to improve the robustness of the
obtained results. The contributions of this paper are twofold: first, we intend to investigate the
contrasts and similarities of the results obtained by distinct MCDA approaches (AHP and MMASSI);
secondly, we expect to enrich the literature of the field with a real-world MCDA case study on a
complex decision making problem since there is a paucity of applied research work addressing real
decision problems faced by organizations.
Keywords:
AHP, decision making, multicriteria decision analysis, multicriteria methodology,
automobile industry.
1. Introduction
In recent years the increasing competitiveness of the global market, as well as the burst of the so-
called Global Financial Crisis forced companies to rethink their processes in order to raise the
levels of efficiency, responsiveness and flexibility. In such contexts, resorting to MCDA to assist
in strategic decision problems can turn out to be a decisive step towards achieving these goals.
MCDA is a formal quantitative approach to aid the decision making process by fostering in
decision makers (DM) the development of a structured thinking of the decision problem at hand.
The main motivation behind the development of this research field is strongly related to the
recognition that human judgments can be limited, distorted and prone to bias, especially when
faced with problems that require the processing and analysis of large amounts of complex
information (Dodgson et al., 2000). Being aware of such hindrances, in the 60's researchers
started to devote themselves to the development of MCDA methods and techniques in an attempt
to overcome the limitations posed by human judgment. Due to its relevance, MCDA quickly
evolved and established itself as an active research field in the 70's. The proposed methods
*
Corresponding author. E-mail: mdbo@inescporto.pt
2
sought to make the decision making process more structured, transparent and efficient. Besides,
the application of MCDA in real-world problems helps increase the confidence of the decision
makers in their decisions, by helping them to reach a solution that complies with their
preferences and system of values. Due to the interactive and iterative nature of MCDA process,
the application of such techniques to real-world scenarios may prove to be a daunting and time
consuming task, which requires a significant endeavor from both analysts (or facilitators) and
decision makers. Therefore, MCDA is more suitable for supporting problems of high complexity
and that may possibly lead to long term impacts (Brito et al., 2010). In this paper, we adopt the
definitions of decision makers and analysts proposed by Belton & Stewart (2002), thus regarding
the decision maker as the one who has the responsibility for the decision, and the analysts as
those who guide and aid the decision makers in the process of reaching a satisfactory decision.
MCDA is a problem solving methodology that organizes and synthesizes the information
regarding a given decision problem in a way that provides the decision maker with a coherent
overall view of the problem. MCDA methods assist DM in the process of identifying the most
preferred action(s), from a set of possible alternative actions (explicitly or implicitly defined),
when there are multiple, complex, incommensurable and often conflicting objectives (e.g.
maximize quality and minimize costs), measured in terms of different evaluation criteria. The
alternative actions distinguish themselves by the extent to which they achieve the objectives,
since usually none of the alternatives has the best performance for all objectives (Dodgson et al.,
2000). Depending on the typology of the MCDA problem at hand, the alternatives can be
implicitly found by solving a mathematical model or they can instead be explicitly known (Lu et
al., 2007). Criteria (also referred to as attributes or objectives) are performance measures
(qualitative or quantitative) that are ranked by the DM, in terms of their perceived importance,
and considered together when appraising the alternatives. By explicitly assessing the
performance of different alternative actions, based on the integration of objective measurement
with subjective value judgment, MCDA techniques unavoidably lead to more efficient and more
informed decisions. The goal of MCDA is not to prescribe the "best" decision to be chosen but to
help decision makers select a single alternative, or a short-list of good alternatives, that best fit
their needs and is coherent with their preferences and general understanding of the problem
(Brito et al., 2010). Usually, the chosen alternative corresponds to the best compromise solution
rather than to an optimal solution.
The views of academics, such as Belton & Stewart (2002), Seydel (2006) and Dooley et al.
(2009), agree that MCDA prompts learning and better understanding of the perspectives of the
DM themselves and the perspectives of the remaining key players involved in the decision
process. Learning and understanding of the problem is mostly achieved by stimulating reflection,
sharing of ideas, and discussion about the problem at hand. This unavoidably leads to an
increased transparency of the decision making process and might hasten the reaching of
consensus. Thus, MCDA can act as a method to document, support, and justify decisions.
Both the academic attention devoted to the field of MCDA and the application of its
methods in real-world decision problems, are a reflection of the advantages of MCDA
approaches in aiding decision making. Bearing this in mind, in this work we aim at presenting a
case study on a real-world decision problem arising in the painting sector of one of Toyota's
3
plants, using the well-known AHP method (Saaty, 1990) and the MCDA method proposed by
Pereira & Sameiro de Carvalho (2005). The contributions of this paper are twofold: firstly, we
intend to investigate the contrasts and similarities of the results obtained by distinct MCDA
approaches (AHP and MMASSI); secondly, we expect to enrich the literature of the field with a
real-world MCDA application to a complex decision-making problem since there is a paucity of
applied research work addressing real decision problems faced by organizations (Dooley et al.
2009). On a different level, it is also our purpose to encourage the adoption of a more structured
thinking in problem solving by the decision makers of this specific company, since previously
their decisions have been mostly made based on intuition and business experience. By
embedding the principles of the scientific method into the decision-making process, decision
makers are able to work through the problem in a more structured way, improving the objectivity
and transparency of the decision process, as well as their commitment to the decision.
Nonetheless, it should be noted that the main motivation behind this study was not to interfere in
the policies and practices of the company, as it would be if an action research scheme was
adopted but, in turn, to stimulate reflection and present new ways to tackle decision problems.
The remainder of this document is organized as follows. Section 2 describes the steps
involved in the deployment process of MCDA. Section 3 describes the MCDA methods used in
this work, namely, AHP and MMASSI. In Section 4, we provide a detailed description of the
application of these methods to a real-world decision-making problem in the painting sector of
one of Toyota’s plants. Section 5 summarizes the paper and offers some concluding remarks.
2. MCDA Process
The deployment of MCDA is a non-linear recursive process comprising several stages. The
number of stages varies according to the adopted MCDA approach, since each one has its own
idiosyncrasies. Nevertheless, it is possible to outline the critical steps of a generic MCDA
process that traverse the great majority of MCDA approaches.
Usually, the first step towards the application of MCDA in real-world problems is related
to both the establishment of a common understanding of the decision context and the
identification of the decision problem. This step involves the decision makers and other key
players that are able to make significant contributions to the MCDA process through the sharing
of their expertise.
The shared perception of the decision context is acquired by means of the
understanding of the objectives of the decision making body and the identification of not only
the set of people that are responsible for the decision, but also those that are likely to be affected
by the decision (Dodgson et al., 2000). The second and third steps of the process comprise the
identification of both the alternatives and the decision criteria that are relevant for appraising
these possible courses of action. According to Dooley et al. (2009), these initial three steps are
usually the most time-consuming tasks of a MCDA process, especially due to their qualitative
nature. The step that follows is the assignment of relative importance weights to the chosen
criteria. These weights can be determined directly (e.g. rating, ranking, swing, trade-off) or
indirectly (e.g. centrality, regression, and interactive). Afterwards, the DM is asked to allot a
subjective score, reflecting his/her opinions, to each one of the identified alternatives according
4
to the criteria deemed important. These scores reflect the judgment of the DM in terms of the
contribution of each alternative to each performance criterion. The information thus obtained is
typically organized into the so-called performance matrix (also referred to as consequence
matrix, options matrix, or simply decision table), where the rows and columns correspond to the
alternatives and the criteria, respectively, and each entry represents the performance of each
alternative against each criterion. The following step of the process involves the summarization
of the information comprised in the performance matrix into a set of multi-criteria scores, one for
each possible course of action. Usually, this is achieved by aggregating (implicitly or explicitly)
the subjective scores of the matrix so as to derive an overall assessment for each alternative that
allows further comparison. Based on these overall scores the set of alternatives is ranked.
Eventually, the process may also involve a sensitivity analysis of the results to changes in scores
or criteria, in order to infer on the robustness of the outcome of MCDA. Finally, the evaluation
and trade-offs involved on the considered alternatives are provided to and discussed with the
DM. In most cases, the final decision taken by the DM does not correspond to the top-ranked
alternative, since they tend to be more concerned with the process of understanding the impact of
each criterion in the ranking of alternatives than in the accuracy of the ranking (Dooley et al.,
2009). Moreover, it is important to note that the results yielded by a MCDA process are not
prone to generalizations, in the sense that they only apply to the set of alternatives that were
evaluated.
3. MCDA Methods
Although several methods have been proposed over the years, here we only describe the AHP
and MMASSI, since these are the ones used in our study. Before presenting these methods, we
first introduce the main schools of thought in the field.
3.1. Dominant Schools of Thought in MCDA
There are two major schools of thought in MCDA that govern the methods proposed in this field:
the French school, represented by the ELimination and (Et) Choice Translating REality
(ELECTRE) family of outranking methods (Roy, 1991); and the American school represented by
the Analytic Hierarchy Process (AHP), proposed by Saaty in the 80’s (Saaty, 1986, 1990). These
dominant schools share the same goal since both are concerned with the problem of assessing a
finite set of alternatives, based on a finite set of conflicting criteria, by a decision making body.
However, they differ in the way they approach the decision problem. According to Lootsma,
methods arising from the French school "model subjective human judgment via partial systems
of binary outranking relations between the alternatives and via a global system of outranking
relations", while methods from the American school build "partial value functions on the set of
alternatives, as well as a global value function" (Lootsma, 1990, page 282). Analogous
distinctions can be made at lower levels of the taxonomy of MCDA methods since even methods
within the same school distinguish themselves in terms of procedures and theoretical
assumptions. These peculiarities should be borne in mind when selecting the most suited MCDA
5
approach to a specific decision problem, due to the lack of consistency of the obtained results. In
other words, the application of different methods to the same decision problem may yield
different results. Hanne (1999) pointed out three important aspects that should be taken into
account when selecting a MCDA method in a real-world decision context, namely:
characteristics of the problem at hand, the method requirements, and the DM requirements. The
characteristics of the problem are related to the categories in which a given MCDM problem
falls. More specifically, if the problem has a continuous set of alternatives, it can be framed as a
Multi-objective Decision Making (MODM) problem, whilst if the decision space is discrete, the
problem falls within the category of Multi-attribute Decision Making (MADM). The proper
identification of the nature of a given decision problem is of utter importance, since some
MCDA methods are only able to handle one of the mentioned types (e.g. interactive approaches
were devised to solve MODM problems, whereas AHP or outranking approaches, are only able
to deal with MADM). Other problem types can be found both in real life and in the literature.
Examples include problems with discrete, integer, or binary and stochastic or fuzzy decision
variables (van Laarhoven & Pedrycz, 1983).
3.2. Methods
3.2.1. AHP
One of the most prevalent and popular approaches for MCDA is AHP. This problem solving
framework was originally developed by the mathematician Thomas Saaty (1986, 1990), in the
late 70’s. AHP belongs to the family of normative methods of the American school of thought.
Albeit the severe criticism and heated debate that AHP has been subjected to by MCDA
academics, its widespread application reflects its generalized acceptance by both the academic
and practitioner communities.
The basic idea behind AHP is to convert subjective assessments of relative importance into
a set of overall scores and weights. The assessments are subjective since, on the one hand they
reflect the perception of the DM, and on the other hand, they are based on pairwise comparisons
of criteria/alternatives. The first step of AHP is to decompose the decision problem into a
hierarchy of subproblems, by arranging the relevant factors of the problem into a hierarchic
structure that descends from an overall goal to criteria, sub-criteria and alternatives, in successive
levels. According to Saaty (1990), the higher levels of the AHP hierarchy should represent the
elements with global character (e.g. the main objective of the decision problem), while the levels
with greater depth should be devoted to the elements that have a more specific nature (e.g.
multiple criteria and alternatives). Resorting to this type of hierarchies provides the DM with an
overall view of the complex relationships inherent in the decision problem, fostering a better
understanding of the problem itself. The second step of the method comprises the elicitation of
pairwise comparison judgments from the decision making body. Here, the DM is asked to assess
the relative importance of criteria with respect to the overall goal, through pairwise comparisons
(e.g. criterion A with criterion B; criterion A with criterion C). The same procedure can be
employed to appraise the alternatives, according to the degree to which they satisfy each
6
criterion. The output of this preference elicitation process is a set of verbal answers of the DM,
which are subsequently codified into a nine-point intensity scale. This semantic scale was
proposed by Saaty (1986) and assumes discrete values from 1 (equally preferable) to 9 (strongly
preferable), where the values 2, 4, 6, and 8 represent intermediate values of preference.
One of the distinguishing characteristics of AHP is the fact that it is grounded in pairwise
comparisons, which are often regarded as straightforward, intuitive and convenient means to
extract subjective information from the DM. However, pairwise comparison strategies rely on
the assumption that the DM is consistent in his/her judgments, which is not always guaranteed in
practice. To measure the degree to which the DM was consistent in his/her responses, a
consistency index is computed for a given matrix. If its value is higher than a specific value
( ) (Saaty, 1986, 1990), then the matrix entries need to be amended since there were
inconsistencies in the DM judgments.
The questions asked to the DM in the previous step of the AHP process aim at achieving
two goals: derive and estimate the priorities or weights of criteria and establish the relative
performance scores for alternatives in each criterion. After the determination of the pairwise
comparisons among criteria, AHP converts the corresponding DM evaluations into a vector of
priorities, by finding the first eigenvector of the criteria matrix. This vector has information
about the relative priority of each criterion with respect to the global goal. The following step of
AHP, which involves the relative importance of criteria, can be performed using two approaches:
one supported on the relative measurement of alternatives and another based on absolute
measurements of these alternatives (Saaty, 1990). In the former approach, separate pairwise
comparisons for the set of alternatives in each criterion (and sub-criterion, if applicable) are
carried out in order to elicit their performance scores. In the latter approach, the alternatives are
simply rated in each criterion, by identifying the grade that best describes them (Saaty, 1990).
Afterwards, a weighting and summing step yields the final results of AHP, which are the
orderings of the alternatives based on a global indicator of priority. The alternative with the
largest value of this global score is the most preferred one.
The main reasons behind the wide applicability of AHP are: its simplicity, since it does not
involve cumbersome Mathematics; the relative ease with which it handles multiple criteria; its
great flexibility, being able to effectively deal with both qualitative and quantitative data; and the
ease of understanding (Kahraman et al., 2003). Besides, the consistency verification operation of
AHP can act as a feedback mechanism for the DM to review and revise the judgments, thus
preventing inconsistencies (Ho et al., 2009). However, despite these advantages, the drawbacks
of AHP instigated a controversial debate among MCDA academics that raises doubts about the
underlying theoretical foundations of the method. The major concerns are closely related to the
rank reversal problem and to the potential inconsistency of the nine-point scale proposed by
Saaty (1986). Rank reversal occurs whenever the addition of one alternative to the initial set of
alternatives modifies the final relative ordering of the alternatives (Goodwin & Wright, 2004).
This situation may lead to different solutions, even if the relative judgments remain unchanged.
Regarding the nine-point scale, it was identified a lack of theoretical foundation between the
points used in the scale and the corresponding verbal description (Goodwin & Wright, 2004).
The effect of the order of the elicitation process can also be understood as a problem because,
7
since criteria priorities are elicited before the performance scores of alternatives, the DM is
induced to make statements about the relative importance of items without knowing, in fact,
what is being compared (Dodgson et al., 2000). According to Dyer (1990), one of the main flaws
of AHP is the ambiguity of the elicitation questions, since they require that the DM explicitly, or
implicitly, determines a reference point in the ratio scale. Seydel (2006) also mentions that the
large number of comparisons required by AHP, especially when dealing with a large number of
criteria and/or alternatives, can turn the pairwise comparisons into a cumbersome and time-
consuming task.
These issues lead us to use another method so that a more confident evaluation and
analysis can be provided to the DM.
3.2.2. MMASSI
Here we perform a comparison of the results yielded by the well-known AHP method and the
ones provided by MMASSI. This way, we are able to increase the level of confidence on the
yielded results, by removing some of the constraints associated to the use of a single method.
MMASSI was first proposed by Pereira & Sameiro de Carvalho (2005) and further extended to
group decision making by Pereira & Fontes (2012). The underpinnings of MMASSI rely on
existing normative methods, which were developed along the lines of the American school of
thought. This methodology was originally devised to aid the decision support process involving a
group of decision makers. However, to fit the scope of our research, we will adapt it to a single-
decision maker (or a consensual group of them). MMASSI can be distinguished from previously
proposed MCDA methodologies in the sense that (a) it provides the DM with a pre-defined set of
criteria that tries to generally cover all the relevant criteria in the field of application; (b) it does
not explicitly requires the presence of a facilitator, or analyst, to guide the DM throughout the
decision process, since it is implemented in an user-friendly and self-explanatory software; and
(c) it uses a continuous scale with two reference levels and thus no normalization of the
valuations is required.
MMASSI methodology encompasses a set of sequential steps that guide the DM through
the several stages of a multi-criteria decision process. MMASSI begins by presenting the DM
with a pre-defined set of criteria, along with their descriptions and suggestions on how to
measure them. These criteria are chosen based on the a priori study of the decision context and
subsequent identification of the features that are consensually considered relevant within its
scope. This provisional family of criteria works as a starting point to guide the DM through the
criteria selection. Nevertheless, it is the DM who defines and assesses the suggested criteria
according to the following range of properties: completeness, redundancy, mutual independence
and operationality (Seydel, 2006). In order to generate the final set of criteria, the DM can refine
the starting set by removing, modifying, or adding criteria. After validating the criteria set, a set
of alternatives is provided by the DM, or the analyst if one is involved, to the MMASSI system.
The following stage comprises the employment of a weighting elicitation technique, namely the
swing-weight procedure proposed by Winterfeldt & Edwards (1986), which sets up the relative
criteria weights according to the preferences expressed by the DM. A fixed continuous scale with
8
seven semantic levels with two references is presented to the DM so as to set up the ground
values based on which he/she assesses each considered alternative against each selected
criterion. The construction of this scale was based on earlier work by Bana e Costa & Vansnick
(1999). The considered levels are the following: Much Worse, Worse, Slightly Worse, Neutral,
Slightly Better, Better and Much Better. This stage implies a mandatory a priori definition of
two reference scale levels, namely, the "Neutral" (or indifference level) and the "Better" levels,
which are to be used to assess each alternative in each criterion. This interval scale is fully
defined by the DM, taking into account the business and organizational context of the analysis,
and it should mirror his/her preferences. Having defined the criteria, the possible courses of
action and a continuous semantic scale, the DM, in the next phase, appraises each alternative by
allotting a semantic level to each criterion. The chosen level should reflect the subjective
preferences and individual judgments of the DM in terms of the extent to which a given
alternative achieves the objectives. The last step of MMASSI involves the computation of an
overall score for each alternative, according to an additive aggregation model, and the
subsequent ranking of the alternatives. Similar to AHP method, the alternative ranked first is
associated to the largest overall score and corresponds to the most preferred alternative.
MMASSI also offers the possibility of performing a sensitivity analysis to assess the robustness
of the preference ranking to changes in the criteria scores and/or the assigned weights.
Sensitivity analysis measures the impact of small perturbations in the variables of the problem
(e.g. criteria scores and criteria weights) in terms of alternatives, by means of the comparison of
the modified ranking with the original one. The closer the rankings, the more robust the method
is. These steps are important to increase the DM’s confidence in the outcome of the multi-criteria
decision analysis.
4. Case Study: Evaluation of Vehicle Painting Plans
The automobile industry has been one of most affected by the global financial downturn, since it
led to a sharp fall on industry sales. Due to this reason, the automobile plant where we carried
out our case study is producing below capacity. Under such adverse circumstances, the
management of the plant felt the need to optimize its processes. Since the painting process is (a)
one of the most complex activities in automobile manufacturing, (b) a bottleneck in this specific
plant, and (c) responsible for the highest costs (e.g. the painting sector costs represent a fraction
of, approximately, 70% of the total expenditures of the entire plant), the plant manager
considered this sector to be the most critical to conduct a MCDA.
The purpose of this case study is twofold. On the one hand, we illustrate the potential of
the application of MCDA for solving a complex decision making problem in the painting sector
of an automobile plant. On the other hand, we analyze the possible vehicle painting plans in
order to provide the DM with an evaluation of the aforementioned plans, as well as with the
identification of the one that contributes the most to the painting process optimization.
In this section we describe the decision problem under consideration, explain how the case
study was carried out, and present the obtained results, by traversing each one of the stages
identified in Section 2.
9
Figure 1. Illustration of the painting process
4.1. Problem Description
The target of our case study is one of the plants of Toyota, located in Ovar, Portugal. The main
function of this plant is to perform the welding, painting, and final assembly of a specific
automotive model. The vehicle parts are delivered to the plant in batches. Each batch includes the
necessary parts to assemble five vehicles. After selecting the necessary parts for production,
according to the production planning, these parts are forwarded to the welding sector. The
welded bodywork of the vehicles is then directed to the painting sector. Since our work focuses
exclusively in this sector, we will later describe the painting process in detail.
The management is interested in optimizing the painting process, which is a bottleneck of
the plant. The only way to improve this process is by optimizing the vehicle painting plans.
These painting plans are defined as a combination of a vehicle type, which can be simple or
mixed, with the number of distinct colors used to paint the vehicles, in a given day. Given this,
the purpose of this case study is, on the one hand, to illustrate the potential of the application of
MCDA for solving a complex decision making problem in the painting sector of an automobile
plant; on the other hand, to provide the DM with an evaluation of the aforementioned painting
plans, as well as with information about the most preferred plan.
4.1.1. Description of the Painting Process
The painting sector comprises a production line which is made up of a series of work stations.
Figure 1 displays the general job flow of the painting process. When the vehicles bodies (or
simply cabins, in this case) are transferred to the paint shop, they are first subjected to a prewash.
The main process begins in the next station, where the surface of the cabins is cleaned and
prepared for the subsequent application of organic coatings through a chemical pretreatment.
Then, the surfaces of the cabins are washed again and further submitted to electrocoating.
Afterwards, they are dried in an oven, with the purpose of baking the coat of paint, and subjected
to a manual inspection.
If any defect is detected, it is repaired by manual sanding. It follows the
application of sealing and PVC to prevent humidity penetration and protect from corrosion. The
sealing is dried in another oven and then the cabins are wiped. The cabins are subsequently
10
subjected to a primer painting, in a spray booth, and dried in an oven. The goal of the primer
painting is to prepare the surface of the cabins to the top-coat application. The operations
performed at work stations 12, 13, 14 and 15 are repeated when applying the top-coat. The
process continues with the manual inspection of the physical aspect of the painted surface. In
case defects are detected, they are repaired through manual sanding and rectification. The
painting process ends with the application of anti-corrosive wax. The painted cabins are then
stocked in a temporary warehouse until being forwarded to final assembly.
Figure 2. The decision hierarchy of the decision problem at hand. At the top level is the main
goal of the decision making. The second level consists of the criteria that contribute to the
overall goal. The third level is comprised of the alternatives that will be evaluated in terms of the
criteria of the second level.
4.2. Data Gathering
The application of MCDA to this decision problem involved the operations manager of the plant
and the painting sector team (henceforth decision maker, or simply DM). Albeit there are several
people involved in the decision making process, they act as if they were a single decision maker,
since the given answers represent the consensual views and preferences of both the manager and
the painting sector team. A number of face-to-face meetings with the DM were convened, so as
to understand the decision context and gather information regarding the decision problem, the
alternatives, and the relevant criteria.
As mentioned before, the goal of the DM is to optimize the painting sector of the plant
through the optimization of the vehicle painting plans. The portfolio of alternatives was
determined by identifying the most frequent painting plans based on daily historical data of the
plant. The analyzed data referred to a time span of six months (June 2012 to December 2012).
Using this procedure we identified eight alternatives, which will be referred to, in this paper, as
PP-A (Painting Plan A), PP-B, through to PP-H. These alternatives were validated by the DM
and are described in Table 3.
11
Table 1. AHP pairwise comparison matrix for criteria and the corresponding criteria weights.
QI
EC
PC
NPV
Criteria Weights
Quality Index (QI)
Energy Consumption (EC)
Paint Consumption (PC)
Number of Painted Vehicles (NPV)
1
1/3
1/7
1/9
3
1
1/2
1/7
7
2
1
1/5
9
7
5
1
0.606
0.23
0.126
0.039
The next step was the selection of the relevant set of criteria to be used to appraise each
one the alternatives. Four quantitative criteria were considered after a brainstorming session with
the DM, namely: the quality index, the energy consumption, the paint consumption and the
number of painted vehicles. The quality index (QI) is given by the average number of defects per
painted vehicle and, as the name implies, it is a proxy for the quality of the performed painting.
Defects can arise as a result of the manual painting process, which is performed by painters, or as
a consequence of the ink quality. Energy consumption (EC) includes both the electricity and the
gas consumption of the painting sector and is measured in kilowatts-hour (kWh). Note that, for
the purpose of this research and for the sake of coherency, gas consumption was converted to
kWh. In turn, paint consumption (PC) reflects the direct cost of painting the vehicles (in terms of
materials), being given by the average ink liters used to paint a given vehicle. The last criterion is
the number of painted vehicles (NPV) per day. More detailed information regarding these criteria
is given in Table 3.
Based on the gathered information, the decision problem is unbundled into its constituent
parts using a AHP hierarchy tree structure comprising three levels (overall goal, criteria, and
alternative painting plans), as depicted in Figure 2. This hierarchical tree has the advantage of
providing an overall view of the complex relationships inherent in the decision problem, thus
easing the understanding of the problem by the DM.
4.3. Elicitation of Criteria Weights
After structuring the decision problem at hand, the DM was asked to assess the relative
importance of the identified criteria based on two different procedures: pairwise comparisons and
swing-weight procedure of Winterfeldt & Edwards (1986). The former is used in the AHP
method, whereas the latter is used in the MMASSI methodology.
These weights are non-negative numbers and independent of the measurement units of the
criteria, and are determined such that higher weights reflect higher importance. The sum of the
normalized weights equals 1, which implies that each criterion can be interpreted according to
their proportional importance.
12
Table 2. Swing-weight scores, as given by the DM, and the corresponding normalized criteria
weights obtained by MMASSI. Criteria Swing
Weights
Quality Index (QI)
Energy Consumption (EC)
Paint Consumption (PC)
Number of Painted Vehicles (NPV)
100
40
20
10
0.588
0.235
0.118
0.059
Total 170 1
4.3.1. AHP
According to AHP, the assignment of weights to the chosen criteria is performed by asking the
DM to form an individual pairwise comparison matrix using the nine-point intensity scale
proposed by Saaty (1990). In this pairwise comparison matrix, the four criteria are compared
against each other in terms of their relative importance, or contribution, to the main goal of the
decision problem. Table 1 shows the pairwise comparison judgments provided by the DM, as
well as the resulting criteria weights. Based on the AHP results, the quality index was deemed
the most important criterion ( ) for the evaluation of the painting plans, followed by
energy consumption ( ) and paint consumption ( ). The least important
criterion is the number of painted vehicles, which was assigned a relative importance of merely
3.9%.
A pairwise comparison matrix is of acceptable consistency if the corresponding
Consistency Ratio (CR) is (Saaty, 1990). Since we obtained , the DM
has been consistent in his judgments and, thus, the obtained criteria weights can be used in the
decision making process.
4.3.2. MMASSI
Contrary to AHP, which relies on pairwise comparisons, MASSI resorts to the swing-weight
procedure to derive criterion weights. According to this procedure, the DM should first identify
the most important criterion, to which a score of 100 is assigned, and then successively allot
relative scores (lower than 100) to the second, third and fourth most important criteria. The given
scores should reflect the DM's order and magnitude of preference and are further normalized so
as to obtain the criteria weights. Table 2 provides both the DM’s scores and the resulting criteria
weights. The comparison of Table 2 with Table 1 shows a considerable similarity between the set
of criteria weights obtained by AHP and the ones returned by MMASSI. This similarity indicates
consistency in the DM’s judgments. Once again, quality index is the criterion with highest
priority, with an influence of 58.8%, followed by the energy consumption ( ), paint
consumption ( ), and finally number of painted vehicles ( ).
13
Table 3. Performance Matrix. The best values observed for each criterion are underlined.
Criteria QI EC PC NPV
Unit
Max/Min # Defects
Min kWh
Min Ink liters
Min # Vehicles
Max
Weights AHP
Weights MASSI 0.606
0.588 0.23
0.235 0.126
0.118 0.039
0.059
PP
-
A
(Simple + 1 Color)
PP-B (Simple + 2 Colors)
PP-C (Simple + 3 Colors)
PP-D (Simple + 4 Colors)
PP-E (Mixed + 1 Color)
PP-F (Mixed + 2 Colors)
PP-G (Mixed + 3 Colors)
PP-H (Mixed + 4 Colors)
3.45
2.1
1.6
3.2
2.1
3.0
2.8
2.5
87
66
60
79
81
73
72
53
2.02
1.85
1.59
1.87
1.55
1.58
1.64
1.56
15
14
30
15
11
21
16
15
4.4. Evaluation and ranking of the alternatives
In this stage, the alternative painting plans are appraised by the DM in terms of their contribution
to the previously stated criteria. To obtain this information, we have asked the DM to provide a
numerical evaluation of the relative performance of each alternative painting plan for each
considered criterion. These numerical evaluations are expressed using the scale adopted by each
MCDA approach (e.g. AHP uses the nine-point intensity scale). To assist the DM in this stage,
we constructed a performance matrix by aggregating the daily data gathered by the painting
sector, for a period of six months (June 2012 to December 2012). This matrix provides objective
information regarding the performance of each alternative on each relevant criterion, and served
as a basis for the DM's evaluation. Upon completion of this stage, the overall score of each
alternative is computed based on an aggregation procedure that takes into account, not only the
alternatives performance evaluation provided by the DM, but also the criteria weights. The final
ranking is generated by sorting the alternatives in decreasing order of the overall scores.
4.4.1. AHP
In this step, the DM is asked to appraise the alternatives by performing separate pairwise
comparisons for the set of alternatives in each criterion. This elicitation process is based on a set
of questions of the general form: “How much more does alternative 1 contributes to the
achievement of criterion A than alternative 2?”. The corresponding verbal answers of the DM are
written down and subsequently codified into the nine-point intensity scale of AHP. These
relative performance scores constitute one of the inputs of a weighting and summing step that
yields the final result of AHP.
14
Table 4. Mandatory reference scale levels of MMASSI, as defined by the DM, for each criterion.
Reference Scale Levels Neutral
Better
Quality Index
Energy Consumption
Paint Consumption
Number of Painted Vehicles
1.8
27
1.79
12
1.6
21
1.66
21
Table 5. Final rankings yielded by AHP and MMASSI methods. The overall scores range from 0
to 100, with a higher score representing a higher level of preference.
AHP Ranking AHP Overall Score MMASSI Ranking MMASSI Overall Score
PP-C 88.29
PP-H 49.53
PP-E 49.08
PP-B 46.92
PP-F 21.9
PP-G 21.71
PP-D 9.09
PP-A 5.16
PP-C 64.64
PP-E 51.05
PP-B 33.69
PP-H 28.09
PP-G 22.95
PP-F 21.72
PP-D 12.96
PP-A 8.72
4.4.2. MMASSI
Regarding MMASSI, the DM was first asked to set, for each criterion, the mandatory reference
levels (neutral and better levels) of MMASSI fixed scale (c.f. Section 3.2.2). These levels are
expressed in the original units of measurement of criteria. The reflection instigated by the need to
define these levels prompted the DM to review and adjust the painting sector goals for each
criterion. The established levels are shown in Table 4. Taking into account these two reference
levels, the DM appraises the set of alternatives, on each criterion, by assigning one of the
following semantic levels to each alternative: Much Worse, Worse, Slightly Worse, Neutral,
Slightly Better, Better or Much Better. In this MCDA step, the major differences between AHP
and MMASSI are the following: (a) in contrast with AHP, MMASSI does not rely on pairwise
comparisons, since each alternative is only assessed in terms of its contribution to each criterion;
(b) instead of using the potentially inconsistent nine-point semantic scale of AHP, MMASSI
relies on a fixed interval scale that is fully defined by the DM.
4.4.3. Comparison of Results
After performing these evaluations, the alternatives were ranked based on a global indicator of
preference. From the analysis of Table 5, we deduce that the most preferred alternative is PP-C,
since it ranks first in both AHP and MMASSI final rankings. Thus, the panting plan with highest
relative merit is the one involving the painting of simple vehicles with three different colors.
From the business viewpoint, this result means that PP-C is the painting plan which contributes
the most to the painting process optimization. In order to compare the similarity of the rankings
returned by the two methods, we compute Kendall's tau rank correlation coefficient (Kendall,
15
1938), denoted as ( ). The obtained value, , indicates the existence of a
significant rank correlation between the AHP and MMASSI final rankings, which means that
both methods yield quite similar results.
4.5. Sensitivity Analysis
Since some steps of the MCDA process can be permeated by subjectivity and uncertainty, we
validated our results by performing a sensitivity analysis in order to determine how the final
ranking of alternatives changes under different criteria weighting schemes. The results for both
AHP and MMASSI have shown that changes in the relative criteria weights did not make any
impact on both the top (i.e. first and second positions) and the bottom (i.e. seventh and eight
positions) of the ranking, although some position shifts were observed in the intermediate
ranking levels (namely, in the third and sixth positions). These conclusions also hold when
introducing considerable changes on the criteria weights, and also for the case in which criteria
have equal priorities.
5. Conclusions
In this paper, we report the application of MCDA to a case study on the automobile industry. The
goal of this case study is to assist the management of the automobile plant in the process of
evaluating the relative merits of alternative painting plans, so as to optimize the painting sector.
This problem is of great relevance for the company, since the painting sector is a bottleneck of
the manufacturing process of the plant. Being aware that MCDA methods are prone to
subjectivity and uncertainty, we resorted to two MCDA methods, namely the well-known AHP
and MMASSI, the MCDA method proposed by Pereira & Sameiro de Carvalho (2005), in order
to increase the confidence, reliability, and robustness of the obtained results.
According to DM's point of view, MMASSI method proved to be swifter and easier to
understand during the preference elicitation stage. This is partly explained by the use of a
continuous scale, rather than semantic one, and by the requirement of a lower number of
evaluations, when compared to AHP. Nevertheless, AHP proved to be more advantageous than
MMASSI for structuring the decision problem. The application of the MCDA methodology
encouraged fruitful discussions and a deeper analysis of the problem peculiarities among the
team. This reflection, along with the process of gathering and summarizing the historical data of
the plant, helped the team to determine the right key performance indicators and the
corresponding target values for the painting sector. Other goals were also achieved, namely, by
providing the team with a framework to address and solve complex problems in a more
structured and scientific way. Regarding the MCDA results, the management found the results
valuable and intends to use the final rankings to enhance the weekly planning of the painting
sector.
We acknowledge that this study is limited in ways that suggest opportunities for future
research. A possible direction would be to solve this decision problem using integrated
16
approaches that combine the strengths of different MCDA methods. We also intend to explore
more formally the distinguishing properties of MMASSI in relation to AHP.
Acknowledgments
The funding by the ERDF (FEDER) through the COMPETE Programme and by FCT within the
project PTDC/EGEGES/099741/2008 is acknowledged. First author was also funded by FCT,
under the PhD grant SFRH/BD/81339/2011. The authors are also thankful to Toyota for
providing the research scenario and the necessary conditions to carry out this case study.
References
Bana e Costa, A. C., & Vansnick, J.-C. (1999). The MacBeth approach: Basic idea, software and
an application. In Meskens, N. and Roubens, M. (Eds.), Advances in decision analysis (pp.
131-157). Dordrecht, The Netherlands: Kluwer Academic Publishers.
Belton, V., & Stewart, T. J. (2002). Multiple Criteria Decision Analysis: An Integrated
Approach. Dordrecht, The Netherlands: Kluwer Academic Publishers.
Brito, T. B., Silva, R. C. d. S., Botter, R. C., Pereira, N. N., & Medina, A. C. (2010). Discrete
event simulation combined with multi-criteria decision analysis applied to steel plant
logistics system planning. In Winter Simulation Conference, pages 2126–2137, Baltimore,
MD, USA.
Dyer, J. S. (1990). Remarks on the Analytic Hierarchy Process. Management Science, 36(3),
249-258.
Dodgson, J., Spackman, M., Pearman, A., & Phillips, L. (2000). Multi-criteria analysis: a
manual. Technical report, Department of the Environment Transport and the Regions.
Dooley, A. E., Smeaton, D. C., Sheath, G. W., & Ledgard, S. F. (2009). Application of multiple
criteria decision analysis in the New Zealand agricultural industry. Journal of Multi-
Criteria Decision Analysis, 16(1-2), 39-53.
Goodwin, P., & Wright, G. (2004). Decision Analysis for Management Judgment. (3rd ed.),
Chichester, United Kingdom: John Wiley and Sons.
Hanne, T. (1999). Meta decision problems in multiple criteria decision making. In Gal, T.,
Stewart, T. J., & Hanne, T. (Eds.), Multicriteria Decision Making: Advances in MCDM
Models, Algorithms, Theory and Applications (pp. 6.1-6.25). Boston, USA: Springer US.
Ho, W., Xu, X., & Dey, P. K. (2009). Multi-criteria decision making approaches for supplier
evaluation and selection: A literature review. European Journal of Operational Research,
202(1), 16-24.
Kahraman, C., Cebeci, U., & Ulukan, Z. (2003). Multi-criteria supplier selection using fuzzy
AHP. Logistics Information Management, 16(6), 382-394.
Kendall, M. G. (1938). A new measure of rank correlation. Biometrika, 30(1/2), 81-93.
Lootsma, F. A. (1990). The French and the American school in multi-criteria decision analysis.
Recherche Opérationelle, 24(3), 263-285.
17
Lu, J., Zhang, G., Ruan, D., & Wu, F. (2007). Multi-objective Group Decision Making: Methods,
Software and Applications with Fuzzy Set Techniques. London, UK: Imperial College
Press.
Pereira, T., & Sameiro de Carvalho, M. (2005). A Decision Support System for Selection of
Information Systems and Information Technologies in distribution Companies. In Kulwant
S. Pawar, Chandra S. Lalwani, José Crespo de Carvalho and Moreno Muffatto (Eds.),
Proceedings of the 10th International Symposium On Logistics: Innovations in Global
Supply Chain Networks. Lisbon, Portugal.
Pereira, T., & Fontes, D. B. M. M. (2012). Group decision making for selection of an
information system in a business context. In DA2PL’2012 Workshop: From Multiple
Criteria Decision Aid to Preference Learning (pp. 74-82). Mons, Belgium.
Roy, B. (1991). The outranking approach and the foundations of ELECTRE methods. Theory
and Decision, 31(1), 49-73.
Hanne, T. (1999). Meta decision problems in multiple criteria decision making. In Gal, T.,
Stewart, T. J., & Hanne, T. (Eds.), Multicriteria Decision Making: Advances in MCDM
Models, Algorithms, Theory and Applications (pp. 6.1-6.25). Boston, USA: Springer US.
Roy, B. (1999). Decision-Aiding Today: What Should We Expect? In Gal, T., Stewart, T. J., &
Hanne, T. (Eds.), Multicriteria Decision Making: Advances in MCDM Models, Algorithms,
Theory and Applications (pp. 1.1-1.35). Boston, USA: Springer US.
Saaty, T. L. (1986). Axiomatic foundation of the Analytic Hierarchy Process. Management
Science, 32(7), 841-855.
Saaty, T. L. (1989). Group Decision Making and the AHP. In Golden, B. L., Wasil, E. A., and
Harker, P. T. (Eds.), The Analytic Hierarchy Process: Applications and Studies (pp. 59-
67). New York, USA: Springer
Berlin Heidelberg.
Saaty, T. L. (1990). How to make a decision: The Analytic Hierarchy Process. European Journal
of Operational Research, 48(1), 9-26.
Seydel, J. (2006). Data Envelopment Analysis for Decision Support. Industrial Management and
Data Systems, 106(1), 81-95.
van Laarhoven, P. J. M. & Pedrycz, W. (1983). A fuzzy extension of Saaty’s Priority Theory.
Fuzzy Sets and Systems, 11(1-3), 199-227.
Winterfeldt, D. V., & Edwards, W. (1986). Decision Analysis and Behavioral Research.
Cambridge, New York, USA: Cambridge University Press.
... Usually, the success of achieving high quality of a component by involving for instance the Autoclave process is compensated by the increased cost and the environmental footprint which are associated to the use of the Autoclave. To achieve an optimum balance between these contradictive design demands Multi Criteria Decision methods (MCD) [43][44][45] need to be involved. For the present Index the Multi Criteria Decision method Analytic Hierarchy Process (AHP) [46,47] has been exploited. ...
... K Q , K env , K cost in Equation (1) stand for dimensionless weight factors which depend on the application and the design restrictions set and reflect the significance of each term for the overall value of the Index. To reduce subjectivity on defining the weight factors they are calculated using the Analytic Hierarchy Process (AHP) which represents a widely used Multi Criteria Decision (MCD) making methodology when a tradeoff between contradicting effects and the desirable result is of concern [44]. ...
... In the literature several MCD making methods are available (such as ELECTRE [50], AHP [51], MAUT [52]). Amongst them the Analytic Hierarchy Process (AHP) has been established as one of the most prevalent and popular ones [44]. The basic idea behind AHP is to convert subjective assessments of relative importance into a set of overall scores and weights. ...
Article
Full-text available
In the present work, a novel holistic component and process optimization index is introduced. The Index is aimed to provide a decision support tool for the optimization of aircraft composite components and manufacturing processes as well as for the selection of the appropriate manufacturing technique of a component when various techniques are considered as manufacturing options. The criteria involved in the index are quality, cost and environmental footprint functions which are considered to be interdependent. In the present concept quality is quantified through measurable technological features which are required for the component under consideration. Cost has been estimated by implementing the Activity Based Concept (ABC) using an in house developed tool. Environmental footprint is assessed by exploiting the ReCiPe method using the ‘open LCA’ software. The weight factor of each of the above criteria in the Index is calculated by using the Multi Criteria Decision (MCD) method Analytic Hierarchy Process (AHP). The Index developed has been applied to support the selection of the appropriate production technique for a typical aeronautical composite part. The alternative manufacturing options considered have been the Automated Fiber Placement (AFP) as well as the classical Autoclave manufacturing technique. By considering quality as the prevailing factor for meeting a decision the index confirms the advantage of the Autoclave process. Yet, by considering the environmental footprint and/or cost to be of equal or higher significance to quality, the implementation of the index demonstrates the clear advantage of AFP process.
... A bibliometric study of MCDA fields reveals that AHP has been adopted in more scientific publications than other MCDA methods [15]. According to the scientific literature, the AHP method has been highly acclaimed [16] by a large number of academics for three main reasons: it can be quickly implemented considering a small number of parameters only [15]; it is supported by user-friendly and commercially software packages [17]; and it ensures high reliability avoiding the 'rank reversals problem' by structuring the decision problem before the decision analysis [18]. All these aspects fit with the nature of the issue addressed in this research work. ...
... The definition of eco-efficiency is given by the ratio between the (added) value of what was manufactured/generated (GDP, income, high quality goods and services offered, etc.) and the (added) impacts of the product or service [21]. For Huppes and Ishikawa [16], eco-efficiency is an instrument to analyze sustainability, in a way that indicates an empirical relationship of economic activities (costs involved) with environmental value (impacts to the environment). Still, it can be pointed out that eco-efficiency presents a role of encouraging business opportunities, allowing companies to become responsible in relation to the environment, while presenting profits [19]. ...
Article
Full-text available
Modelling eco-efficiency is becoming a wide field of research shown by a large body of scientific literature on the subjects concerned. This paper, after performing a systematic literature review (SLR) to identify existing models for process eco-efficiency assessments, provides a methodology , based on the analytic hierarchy process (AHP) method, for choosing the eco-efficiency assessment model to be used for a given application (process, product, or service). For the SLR, papers from the databases Scopus, Web of Science and Science Direct were used. Forty articles were considered for this study, using as the main selection criterion articles that present an eco-efficiency assessment model, since the purpose was to survey the types of existing models that are used to assess processes, products or services for eco-efficiency. With the systematic review carried out, it was possible to identify the types of models that exist and how they are used in different sectors, always aiming to identify if what was analyzed is eco-efficient and what points need to be improved. The proposed AHP-based methodology was applied to a numerical model to outline how to apply the methodology. The approach was easy to use and effective in identifying the proper eco-efficiency model.
... MCDM is an important method in expert systems and operation research entailing several decision alternatives and criteria [19][20][21][22][23][24][25], and it involves structuring, planning and solving decision problems by utilising numerous criteria [26][27][28][29][30]. The utilisation of MCDM is rapidly gaining popularity owing to its ability to improve the quality of decisions through a more explicit, rational and efficient process compared with the conventional approaches [31][32][33][34][35]. Over the decades, scholars have developed multiple MCDM methods and approaches to solve multiple discrete problems in different fields [36][37][38][39][40]. MCDM is utilised to address the essential requirement of locating the most eligible alternative(s) amongst a set of alternatives based on the selected criteria, in which this set of alternatives shares the same decision criteria as the decision matrix (DM) to solve the decision-making problems [41][42][43][44]. ...
Article
Full-text available
Context The vaccine distribution for the coronavirus disease of 2019 (COVID-19) is a multicriteria decision-making (MCDM) problem based on three issues, namely, identification of different distribution criteria, importance criteria and data variation. Thus, the Pythagorean fuzzy decision by opinion score method (PFDOSM) for prioritising vaccine recipients is the correct approach because it utilises the most powerful MCDM ranking method. However, PFDOSM weighs the criteria values of each alternative implicitly, which is limited to explicitly weighting each criterion. In view of solving this theoretical issue, the fuzzy-weighted zero-inconsistency (FWZIC) can be used as a powerful weighting MCDM method to provide explicit weights for a criteria set with zero inconstancy. However, FWZIC is based on the triangular fuzzy number that is limited in solving the vagueness related to the aforementioned theoretical issues. Objectives This research presents a novel homogeneous Pythagorean fuzzy framework for distributing the COVID-19 vaccine dose by integrating a new formulation of the Pythagorean fuzzy-weighted zero-inconsistency (PFWZIC) and PFDOSM methods. Methods The methodology is divided into two phases. Firstly, an augmented dataset was generated that included 300 recipients based on five COVID-19 vaccine distribution criteria (i.e., vaccine recipient memberships, chronic disease conditions, age, geographic location severity and disabilities). Then, a decision matrix was constructed on the basis of an intersection of the ‘recipients list’ and ‘COVID-19 distribution criteria’. Then, the MCDM methods were integrated. An extended PFWZIC was developed, followed by the development of PFDOSM. Results (1) PFWZIC effectively weighted the vaccine distribution criteria. (2) The PFDOSM-based group prioritisation was considered in the final distribution result. (3) The prioritisation ranks of the vaccine recipients were subject to a systematic ranking that is supported by high correlation results over nine scenarios of the changing criteria weights values. A comparison with previous work also proved the efficiency of the proposed framework. Conclusion The findings of this study are expected to contribute to ensuring equitable protection against COVID-19 and thus help accelerate vaccine progress worldwide.
... [91][92][93][94] Further, MCDM has the abilities to improve the quality of a decision via e±cient and rational processes more than traditional methods. 95 Besides, MCDM intends to classify and rank proper alternatives among a set of available alternatives. [96][97][98][99][100] Relevant alternatives are scored based on prior goals. ...
Article
Increasing demand for open-source software (OSS) has raised the value of efficient selection in terms of quality; usability is an essential quality factor that significantly affects system acceptability and sustainability. Most large and complex software packages partitioned across multiple portals and involve many users — each with their role in the software package; those users have different perspectives on the software package, defined by their knowledge, responsibilities, and commitments. Thus, a multi-perspective approach has been used in usability evaluation to overcome the challenge of inconsistency between users’ perspectives; the inconsistency challenge would lead to an ill-advised decision on the selection of a suitable OSS. This study aimed to assist the public and private organizations in evaluating and selecting the most suitable OSS. The evaluation of the OSS software packages to choose the best one is a challenging task owing to (a) multiple evaluation criteria, (b) criteria importance, and (c) data variation; thus, it is considered a sophisticated multi-criteria decision making (MCDM) problem; moreover, the multi-perspective usability evaluation framework for OSS selection lacks in the current literature. Hence, this study proposes a novel multi-perspective usability evaluation framework for the selection of OSS based on the multi-criteria analysis. Integration of best-worst method (BWM) and VIKOR MCDM techniques has been used for weighting and ranking OSS alternatives. BWM is utilized for weighting of evaluation criteria, whereas VIKOR is applied to rank OSS-LMS alternatives. Individual and group decision-making contexts, and the internal and external groups aggregation were used to demonstrate the efficiency of the proposed framework. A well-organized algorithmic procedure is presented in detail, and a case study was examined to illustrate the validity and feasibility of the proposed framework. The results demonstrated that BWM and VIKOR integration works effectively to solve the OSS software package benchmarking/selection problems. Furthermore, the ranks of OSS software packages obtained from the VIKOR internal and external group decision making were similar; the best OSS-LMS based on the two ways was ‘Moodle’ software package. Among the scores of groups in the objective validation, significant differences were identified; this indicated that the ranking results of internal and external VIKOR group decision making were valid, which pointed to the validation of the framework.
... In addition, because COVID-19 classification techniques have to consider multiple criteria, it considered as a difficult and challenging task in time and error rate in the dataset which also could be significantly important in the COVID-19 classification. In addition, each decision-maker assigns a different weight for all these previous criteria [53]. ...
Article
Full-text available
This study presents a systematic review of artificial intelligence (AI) techniques used in the detection and classification of coronavirus disease 2019 (COVID-19) medical images in terms of evaluation and benchmarking. Five reliable databases, namely, IEEE Xplore, Web of Science, PubMed, ScienceDirect and Scopus were used to obtain relevant studies of the given topic. Several filtering and scanning stages were performed according to the inclusion/exclusion criteria to screen the 36 studies obtained; however, only 11 studies met the criteria. Taxonomy was performed, and the 11 studies were classified on the basis of two categories, namely, review and research studies. Then, a deep analysis and critical review were performed to highlight the challenges and critical gaps outlined in the academic literature of the given subject. Results showed that no relevant study evaluated and benchmarked AI techniques utilised in classification tasks (i.e. binary, multi-class, multi-labelled and hierarchical classifications) of COVID-19 medical images. In case evaluation and benchmarking will be conducted, three future challenges will be encountered, namely, multiple evaluation criteria within each classification task, trade-off amongst criteria and importance of these criteria. According to the discussed future challenges, the process of evaluation and benchmarking AI techniques used in the classification of COVID-19 medical images considered multi-complex attribute problems. Thus, adopting multi-criteria decision analysis (MCDA) is an essential and effective approach to tackle the problem complexity. Moreover, this study proposes a detailed methodology for the evaluation and benchmarking of AI techniques used in all classification tasks of COVID-19 medical images as future directions; such methodology is presented on the basis of three sequential phases. Firstly, the identification procedure for the construction of four decision matrices, namely, binary, multi-class, multi-labelled and hierarchical, is presented on the basis of the intersection of evaluation criteria of each classification task and AI classification techniques. Secondly, the development of the MCDA approach for benchmarking AI classification techniques is provided on the basis of the integrated analytic hierarchy process and VlseKriterijumska Optimizacija I Kompromisno Resenje methods. Lastly, objective and subjective validation procedures are described to validate the proposed benchmarking solutions.
... 60,61,[146][147][148] The use of MCDM is rapidly increasing owing to its ability to enhance the quality of decisions by making process decisions more e®ective, rational, and explicit compared with standard procedures. 62,[132][133][134][135][136] The aims of MCDM are to (1) help data miners select the best alternative, 137 (2) categorize the viable alternative among a set of available alternatives, 138 and (3) rank the performance of the alternatives in descending order. 63,64 Appropriate alternative(s) are scored. ...
Article
The increasing demand for image dehazing-based applications has raised the value of efficient evaluation and benchmarking for image dehazing algorithms. Several perspectives, such as inhomogeneous foggy, homogenous foggy, and dark foggy scenes, have been considered in multi-criteria evaluation. The benchmarking for the selection of the best image dehazing intelligent algorithm based on multi-criteria perspectives is a challenging task owing to (a) multiple evaluation criteria, (b) criteria importance, (c) data variation, (d) criteria conflict, and (e) criteria tradeoff. A generally accepted framework for benchmarking image dehazing performance is unavailable in the existing literature. This study proposes a novel multi-perspective (i.e., an inhomogeneous foggy scene, a homogenous foggy scene, and a dark foggy scene) benchmarking framework for the selection of the best image dehazing intelligent algorithm based on multi-criteria analysis. Experiments were conducted in three stages. First was an evaluation experiment with five algorithms as part of matrix data. Second was a crossover between image dehazing intelligent algorithms and a set of target evaluation criteria to obtain matrix data. Third was the ranking of the image dehazing intelligent algorithms through integrated best–worst and VIseKriterijumska Optimizacija I Kompromisno Resenje methods. Individual and group decision-making contexts were applied to demonstrate the efficiency of the proposed framework. The mean was used to objectively validate the ranks given by group decision-making contexts. Checklist and benchmarking scenarios were provided to compare the proposed framework with an existing benchmark study. The proposed framework achieved a significant result in terms of selecting the best image dehazing algorithm.
... MCDM is also used for structuring, planning and implementing decisions [70][71][72][73][74][75][76]. Given its ability to improve the quality of decisions through a highly reliable and reasonable decision making in contrast to standard procedures, MCDM has been increasingly employed in the literature [77][78][79][80][81]. MDCM has three goals, namely: (1) to assist in the selection of the best possible alternative [82][83][84], (2) to identify the practicable alternative amongst a number of alternatives [85][86][87] and (3) to rank the alternatives in a descending order based on their performance [88][89][90][91][92]. ...
Article
Full-text available
Given the rapid development of dehazing image algorithms, selecting the optimal algorithm based on multiple criteria is crucial in determining the efficiency of an algorithm. However, a sufficient number of criteria must be considered when selecting an algorithm in multiple foggy scenes, including inhomogeneous, homogenous and dark foggy scenes. However, the selection of an optimal real-time image dehazing algorithm based on standardised criteria presents a challenge. According to previous studies, a standardisation and selection framework for real-time image dehazing algorithms based on multi-foggy scenes is not yet available. To address this gap, this study proposes a new standardisation and selection framework based on fuzzy Delphi (FDM) and hybrid multi-criteria analysis methods. Experiments are also conducted in three phases. Firstly, the image dehazing criteria are standardised based on FDM. Secondly, an evaluation experiment is conducted based on standardised criteria and nine real-time image dehazing algorithms to obtain a multi-perspective matrix. Third, entropy and VIKOR methods are hybridised to determine the weight of the standardised criteria and to rank the algorithms. Three rules are applied in the standardisation process to determine the criteria. To objectively validate the selection results, mean is applied for this purpose. The results of this work can be taken into account in designing efficient methods and metrics for image dehazing.
Thesis
Full-text available
Multi-Criteria Decision-Making (MCDM) is about techniques including software for making decisions when multiple criteria (or objectives) essential to be considered together, in order to rank or choose among the alternatives being evaluated. The MCDM as well as multiple-criteria decision analysis (MCDA) is a sub-discipline of operations research that openly assesses various conflicting criteria in decision making (both in daily life and in settings for example, medicine business and government.
Article
The selection of software programmer applicants based on multiperspective evaluation criteria (grade point average (GPA) and soft skills of the applicants) is needed instead of an interview because an interview does not necessarily lead to hiring the best candidate amongst the applicants. The selection of a suitable software programmer is considered a challenging task owing to the following factors: (1) data variation, (2) multiple evaluation criteria and (3) criterion importance. A general framework for the selection of the best software programmer applicants is not available in the existing literature. The present study aims to propose a novel multiperspective hiring framework based on multicriteria analysis to select the best software programmer amongst several applicants. A decision matrix (DM) is constructed for the selection of the best programmer applicants according to multiple criteria, namely, structured programming, object-oriented programming, data structure, database system and courseware engineering. Each criterion includes two parameters, namely, GPA and soft skills, and these criteria cross over with programmer applicants as alternatives. The standard and expert opinion of the Software Engineering Body of Knowledge is used to distribute the criteria in the DM. The two commonly used techniques of multicriteria decision-making are analytic hierarchy process (AHP) for weighing the criteria and technique for order performance by similarity to ideal solution (TOPSIS) for ranking the alternatives (programmer applicants). The data used in this study include 60 software engineering students who graduated in 2016 from Universiti Pendidikan Sultan Idris. Results show that integrating multilayer analytic hierarchy process (MLAHP) and group TOPSIS are effective for solving applicant selection problems. Group TOPSIS uses different contexts — internal and external aggregation — and indicates similar results. Objective validation is used for the ranking of the results, which are equally divided into four parts. Furthermore, the applicants are systematically ranked. This study benefits application software, system software and computer programming tool companies by providing a method that improves software quality whilst reducing time and cost in the selection process.
Article
Supplier evaluation and selection problem has been studied extensively. Various decision making approaches have been proposed to tackle the problem. In contemporary supply chain management, the performance of potential suppliers is evaluated against multiple criteria rather than considering a single factor-cost. This paper reviews the literature of the multi-criteria decision making approaches for supplier evaluation and selection. Related articles appearing in the international journals from 2000 to 2008 are gathered and analyzed so that the following three questions can be answered: (i) Which approaches were prevalently applied? (ii) Which evaluating criteria were paid more attention to? (iii) Is there any inadequacy of the approaches? Based on the inadequacy, if any, some improvements and possible future work are recommended. This research not only provides evidence that the multi-criteria decision making approaches are better than the traditional cost-based approach, but also aids the researchers and decision makers in applying the approaches effectively. (C) 2009 Elsevier B.V. All rights reserved.
Article
In the first part of this paper, we describe the main features of real-world problems for which the outranking approach is appropriate and we present the concept of outranking relations. The second part is devoted to basic ideas and concepts used for building outranking relations. The definition of such outranking relations is given for the main ELECTRE methods in Part 3. The final part of the paper is devoted to some practical considerations.
Chapter
This paper focuses on the application of the Analytic Hierarchy Process in a group setting. In particular, we present observations and suggestions that are intended to help in the planning and execution of a group decision-making effort in which AΗΡ plays a major role.
Article
The problem of selecting a method for solving an MCDM problem is dicussed. This problem called meta decision problem can be formulated and solved in different ways. The most common approach is to define it as a problem of choosing one method from a finite set of methods which are evaluated according to several criteria. This leads to a formalization of the meta decision problem as an MCDM problem itself. Scalar evaluations of methods can help to avoid the meta decision problem becoming too complex. The problem of assessing the parameter(s) of an MCDM problem is similar to the meta decision problem above and can be interpreted as the problem to design an MCDM method. This leads to a formalization of the meta decision problem as a scalar parameter optimization problem. Information for solving meta decision problems can be submitted by a decision maker or is given as a data file originating, e.g., from prior decision making processes. Thus a meta decision problem can be solved in an interactive framework or through machine learning.