Figure 4 - uploaded by Yves De Smet
Content may be subject to copyright.
Shanghai ranking: variation of the rank of every alternative when another alternative is deleted (see online version for colours)
Source publication
The multicriteria methods based on pairwise comparisons suffer from possible rank reversal occurrences when the set of alternatives is modified. We study this distinctive feature in the scope of the PROMETHEE I and II methods. First, empirical tests are conducted on the basis of artificial datasets in order to quantify the likelihood of rank revers...
Context in source publication
Similar publications
Every complex problem now days require multicriteria decision making to get to the desired solution. Numerous Multi-criteria decision making (MCDM) approaches have evolved over recent time to accommodate various application areas and have been recently explored as alternative to solve complex software engineering problems. Most widely used approach...
Citations
... In [18,Abstract], rank reversal is called ''one of the most important challenges'' in MCDM. Rank reversal of one or another type occurs for virtually every MCDM method, and its implications are still highly debated: One may compare, for example, [5,26] for surveys and [10] for TOPSIS, [20,21] for AHP, [8,28] for ELECTRE and [7,22,27] for PROMETHEE. New methods such as COMET and SPOTIS were recently proposed which are rank reversal free (see [18,29]). ...
A basic problem in multi-criteria decision-making (MCDM) is to find a ranking for alternatives which are not directly comparable with each other. A number of different methods exists. In this paper, a new ranking method is proposed by turning a statistical function into an MCDM ranking function. In particular, cone distribution functions from multivariate statistics are used as ranking functions and their features are investigated. Our findings demonstrate that this procedure can be considered as an upgrade of the weighted sum ranking insofar as it absorbs a whole collection of weighted sums at once instead of fixing a particular one in advance. The new ranking–in contrast to a pure weighted sum ranking–is also able to detect “non-convex” parts of the Pareto frontier. The rank reversal phenomenon is studied, and it is explained why it might even be useful for analyzing the ranking procedure. The ranking is extended to sets providing unary indicators for set preferences which establishes the link between set optimization methods and set-based multi-objective optimization. This also provides a new tool for evaluating the outcomes of evolutionary algorithms for multi-criteria optimization. The proposed method has implications for preferences learning and categorization of multi-dimensional data points, both with respect to an underlying non-complete order relation.
... Rank reversal (RR) is the situation in which a change occurs in the ranking results after adding or removing alternatives to a predetermined group (Lootsma 1993;Saaty and Sagir 2009). Although the RR problem was first examined in the literature using the AHP method, research on this issue has increased rapidly, and it has been investigated for various other MCDA methods, including TOPSIS and PROMETHEE (Triantaphyllou 2001;Kong 2011;Verly and De Smet 2013). Adding and removing different factors can change the rankings significantly, which raises questions about the efficiency of MCDA methods. ...
Financial performance analysis is of vital importance those involved in a business (e.g., shareholders, creditors, partners, and company managers). An accurate and appropriate performance measurement is critical for decision-makers to achieve efficient results. Integrated performance measurement, by its nature, consists of multiple criteria with different levels of importance. Multiple Criteria Decision Analysis (MCDA) methods have become increasingly popular for solving complex problems, especially over the last two decades. There are different evaluation methodologies in the literature for selecting the most appropriate one among over 200 MCDA methods. This study comprehensively analyzed 41 companies traded on the Borsa Istanbul Corporate Governance Index for 10 quarters using SWARA, CRITIC, and SD integrated with eight different MCDA method algorithms to determine the position of Turkey's most transparent companies in terms of financial performance. In this study, we propose "stock returns" as a benchmark in comparing and evaluating MCDA methods. Moreover, we calculate the "rank reversal performance of MCDA methods". Finally, we performed a "standard deviation" analysis to identify the objective and characteristic trends for each method. Interestingly, all these innovative comparison procedures suggest that PROMETHEE II (preference ranking organization method for enrichment of evaluations II) and FUCA (Faire Un Choix Adéquat) are the most suitable MCDA methods. In other words, these methods produce a higher correlation with share price; they have fewer rank reversal problems, the distribution of scores they produce is wider, and the amount of information is higher. Thus, it can be said that these advantages make them preferable. The results show that this innovative methodological procedure based on 'knowledge discovery' is verifiable, robust and efficient when choosing the MCDA method.
... Keyser and Peeters (1996) have shown that, such as most other outranking methods, Promethee methods suffer from the rank reversal phenomenon: when one alternative is removed or added to the dataset, the respective order in the ranking of two other alternatives can be reversed. This phenomenon has however been mitigated in (Mareschal et al., 2008), (Verly and Smet, 2013) and more recently in . We can therefore assume that even if ...
Multicriteria decision aid consists of helping decision makers to compare (rank, choose, sort, etc.) different alternatives which are evaluated on conflicting criteria. In the last decades, numerous decision aid methods have been developed. Three main categories of decision aid methods are usually considered: the aggregating, interactive and outranking methods. While aggregation methods produce a complete ranking of the set of alternatives, outranking methods usually allow some pairs of alternatives to remain incomparable. This happens either if the two alternatives present some strong conflicting information or if there are not enough elements in the decision problem to state the preference or indifference between them. A well‐known family of outranking procedures are the Promethee methods. The aim of this work is first to provide an analysis of the incomparability relation produced by Promethee I. From our point of view, some shortcomings of this incomparability relation are presented. Then, a new method based on the comparison of weighted coalitions of mono‐criterion net flow score differences, called Promethee γ, is proposed. Promethee γ and Promethee i are then further compared.
... The evaluation of the causes of the RR phenomenon in the ELECTRE II method was investigated by Liu and Ma (2021). The evaluation of RR phenomenon in PROMETHEE I and PROMETHEE II methods was investigated by Verly and De Smet (2013). They reported that there are RR cases in PROMETHEE methods and under certain conditions, the RR phenomenon is prevented in this method. ...
... In general, there are many weaknesses in most previous studies related to RR phenomenon. First, the contribution of RR phenomenon studies in the literature related to decision-making methods including AHP (Majumdar et al., 2021;Maleki and Zahir, 2013;Wang and Elhag, 2006), TOPSIS (Aires and Ferreira, 2019;Tiwari and Kumar, 2021;Yang, 2020), ELECTRE (Costa, 2016;Figueira and Roy, 2009;Liu and Ma, 2021;Wang and Triantaphyllou, 2008), PROMETHEE (Mareschal et al., 2008;Roland et al., 2012;Verly and De Smet, 2013), VIKOR (Agrawal, 2021;Ceballos et al., 2018;Mousavi-Nasab and Sotoudeh-Anvari, 2018;Yang and Wu, 2020), MOORA (Agrawal, 2021;Mufazzal et al., 2022) was very high. The main reason is that these methods are old and the phenomenon of RR has been reported in these methods more than any other method in the literature. ...
In the present study, a novel approach based on multi-criteria analysis was proposed to prioritize PM mitigation strategies from economic, environmental and social aspects. Multi-criteria decision making (MCDM) methods are suitable tools for prioritizing strategies on various topics. However, it has been observed that adding/removing strategies has led to a change in preference which is called rank reversal. The present study was aimed with three objectives: (i) evaluating the rank reversal phenomenon in MCDM methods frequently used in environmental topics, (ii) introducing a novel approach based on wins in league (WIL) in order to reduce the rank reversal in MCDM methods and (iii) a new application of this method in prioritizing PM mitigation strategies in arid urban areas. Qualitative results showed that in studies in the field of environment, the rank reversal has occurred frequently in classical MCDM methods but its occurrence was prevented in the proposed method. Quantitative evaluation also showed that the probability of rank reversal occurrence in Technique For Order Performance By Similarity To Ideal Solution (TOPSIS), Additive Ratio ASsessment (ARAS), Weighted Aggregated Sum Product Assessment (WASPAS), Evaluation Based On Distance From Average Solution (EDAS), Combinative Distance-Based Assessment (CODAS) and Measurement Alternatives and Ranking According to Compromise Solution (MARCOS) methods was 36.4%, 32.8%, 21.2%, 31.6%, 36.1% and 29.9%, respectively. However, the probability of rank reversal occurrence in the proposed approach was almost 90% less than classical MCDM methods. Evaluation of PM mitigation strategies in arid urban areas has also shown that urban green belt plan is the most appropriate strategy.
... To check if removal of an indicator caused final rank reversals (Aires and Ferreira, 2018;De Smet, 2013), we performed a leave-one-out analysis removing each indicator stepwise, re-running PROMETHEE II analysis, and comparing resulting Phi values. ...
In view of the increasing demand for ecosystem services (ESs) and rising pressures from global change on mountain lakes, sustainable management strategies are increasingly needed. However, integrative assessments of ESs provided by small mountain lakes are lacking. This study adopted an integrative ES valuation perspective, applying multi-criteria decision analysis (MCDA) to four exemplary mountain lakes in South Tyrol, Italy. With the involvement of stakeholders, we identified five ESs of particular importance: provision of habitat, aesthetics, surface water, recreation, and representation. These ESs were weighted with respect to their perceived importance and quantified based on multi-metric indicators integrating biophysical and socio-cultural/economic aspects. The current ES provision of mountain lakes was then evaluated by applying PROMETHEE II. We showed that the quantity of ESs varied with distinct socio-ecological characteristics. Well accessible and larger lakes currently provide larger amounts of ES in high demand, such as habitat provision, aesthetics, and recreation, but remarkably, smaller and remote lakes also provide notable amounts of ES aesthetics. Our integrative approach provides insights to support sustainable ES management of mountain lakes considering future challenges that are potentially applicable in other regional contexts.
... Mareschal et al. analyze the PROMETHEE II method and clarify different conditions under which the ranking of this method might suffer rank reversal (Mareschal, De Smet, & Nemery, 2008). Verly et al. also observed the rank reversal problem in PROMETHEE I and II (Verly & De Smet, 2013). They showed that the rank reversal phenomenon occurs when the flow score differences are relatively small. ...
Decision-making appears as a complex and challenging task when it requires finding the most suitable alternative among the numerous alternatives in the presence of multiple, usually conflicting criteria. At the same time, stakeholders expect a simple, transparent, and traceable decision-making method. Multi-Criteria Decision-Making (MCDM) methods rank the alternatives considering multiple criteria. The rank reversal problem is an important issue in most existing conventional MCDM methods. This paper proposes a new alternative by alternative comparison-based MCDM Method (ABAC) that addresses the rank reversal problem. We prove that ABAC is free from the rank reversal problem. To illustrate and validate ABAC, we have taken the cloud service selection problem as an application. Further, to show the effectiveness of ABAC, we have provided several case studies covering various domains. We perform several experiments by simulating the ABAC method. We have compared ABAC and existing MCDM methods. The experimental results support that the ABAC method is a rank reversal free MCDM method. We also carry out sensitivity analysis for ABAC. Salient features of ABAC over existing MCDM methods are (i) it is simple; (ii) it is rank reversal free; (iii) it is more scalable.
... In such cases, it is expected that the MCDM method will not show a serious change in the ranking of alternatives. This phenomenon is called the "rank reversal problem" and has been given considerable attention in the literature (Verly and De Smet, 2013;García-Cascales and Lamata, 2012). One of the ways to test the validity of the results obtained from the model for decision-making is to create dynamic matrices and then analyze the solutions that the model offers under the newly created conditions. ...
The aim of this study is to propose a model of stability and sensitivity for the studies used in the implementation of multi-criteria decision making (MCDM). In this context, it is proposed to use the steps "sensitivity analysis based on the variation of criterion weight, sensitivity analysis based on sequence reversal feature and comparison analysis with results from different sorting methods" for stability and sensitivity analysis. In the implementation part of the method, the Fragile Five countries were used as criteria for the unemployment rate, state budget, GDP growth, inflation, current account balance, risk premium for these countries. The criteria were weighted with MEREC and the ordering of alternatives was carried out with WISP. In the application phase of the method, it was found that the model was sensitive to changes in weight coefficients with different weights assigned to criteria over 22 scenarios. It has been observed that the model provides valid results in a dynamic environment through 4 different scenarios created in the sequence inversion step of the model. For the reliability of the MEREC-WISP-based model, a ranking comparison was made with some commonly used methods such as PIV, CoCoSo, COPRAS, WEDBA, TOPSIS and SAW and the results were found to have a high correlation.
... This approach brings several important benefits. Probably the most important one is that this method does not suffer from rank-reversal (this issue is thoroughly explored by Verly and De Smet (2013) in relation to PROMETHEE) because the final values of the alternatives are mutually independent. This gives rise to another side benefit: if an alternative is changed, removed, or added, it is not necessary to recalculate again all φ values, which is a very welcomed feature for the problems solved repeatedly. ...
PROMETHEE method is a very popular quantitative method of decision-making with many benefits. However, the evaluation of alternatives in the original PROMETHEE method is derived only from differences in values, i.e., regardless the performance values themselves. In some situations, ignoring these values can distort the final results. This paper brings several examples of such situations, for which the original PROMETHEE fails and does not bring reliable results. Ishizaka and Resce (Soft Comput 22:7325–7338, 2018) have recently introduced the modification of PROMETHEE which considers the performance values, but also changed substantially the logic of the ranking algorithm. The aim of this paper is to modify the original PROMETHEE method to make it possible to include the performance values, without losing any main benefit of the original method and with keeping the original logic of the algorithm based on pair-wise comparisons. Two particular preference functions’ types are proposed for the proposed extension (Gaussian function and strictly concave function), whose choice depends on the performance of the worst-performing alternative under consideration. In addition, the new algorithm is provided also in the fuzzy environment, which is useful if the decision-maker is not able to set the input parameters of the preference function precisely. Both the deterministic and fuzzy extensions are demonstrated using numerical examples. The results show that the final ranking can be strongly influenced by the level of performance. Moreover, the fuzzy extension brings richer information through the natural interpretation provided by possibility and necessity measures if the parameters of the preference functions are imprecise.
... This approach makes the SIMUS approach more attractive for proposing a hybrid framework in the case of urban mobility planning compared to other MCDM methods such as AHP [36], MOORA [39], ELECTRE [40,41], PROMETHEE [42], SAW [41] or TOPSIS [43], for example. ...
Sustainable Urban Mobility Plans (SUMP) are increasingly popular planning tools in cities with environmental issues where numerous actions are usually proposed to reduce pollution from urban transport. However, the diagnosis and implementation of these processes requires broad consensus from all stakeholders and the ability to fit them into urban planning in such a way that it allows the proposals to become realistic actions. In this study, a review of the sustainable urban mobility plans of 47 cities in Spain during the last 15 years has been carried out, analyzing both the diagnosis and proposal of solutions and their subsequent implementation. From the results obtained, a new framework based on a structured hybrid methodology is proposed to aid decision-making for the evaluation of alternatives in the implementation of proposals in SUMP. This hybrid methodology considers experts’ and stakeholders’ opinion and applies two different multi-criteria decision making (MCDM) methods in different phases to present two rankings of best alternatives. From that experience, an analysis based on the MCDM methods called ‘Sequential Interactive Modelling for Urban Systems (SIMUS)’ and weighted sum method (WSM) was applied to a case study of the city of Cartagena, a southeastern middle-size city in Spain. This analytic proposal has been transferred to the practical field in the SUMP of Cartagena, the first instrument of this nature developed after COVID-19 in Spain for a relevant city. The results show how this framework, based on a hybrid methodology, allows the development of complex decision mapping processes using these instruments without obviating the need to generate planning tools that can be transferred from the theoretical framework of urban reality.
... It is proved that the proposed approaches are valid and they can be used to analyze RR problems in the ELECTRE II method. Verly and De Smet (2013) studied RR problems in PROMETHEE I and PROMETHEE II methods. They performed empirical tests on artificial datasets to find the probability of RR by deleting a non-discriminating criterion, deleting and adding a copy of existing irrelevant alternative. ...
... WASPAS method is used for ranking the alternatives in randomly generated original decision matrix and the decision matrix where a non-discriminating criterion is removed. The weights of the criteria for the first decision problem are generated, as explained previously, and weights for the second decision problem are adjusted by normalizing the weights of the remaining criteria, which is used in Verly and De Smet (2013). If the best alternatives of both rankings are different, it is concluded that the method has the corresponding RR type. ...
Multiple attribute decision-making (MADM) methods are commonly employed to assist decisions for selecting the best alternative according to conflicting criteria in complex decision situations. Although MADM methods have proven to be very useful, in some dynamic decision cases they may cause faulty results due to the “rank reversals” problem. Numerous MADM methods such as Analytic Hierarchy Process (AHP), Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), Preference Ranking Organization Method for Enrichment Evaluations (PROMETHEE), and several others well-known methods proven to have rank reversal problems. In this paper, we study the rank reversal problem for a recent MADM method that is known as Weighted Aggregated Sum Product Assessment (WASPAS). As far as we know, there is no study that considers the rank reversal problem in the WASPAS. In this paper, rank reversal problems in WASPAS are analyzed empirically by considering different types of rank reversals. After detailed computational experiments, we show that rank reversal problems also exist in WASPAS when classical normalization techniques are utilized. We also show through extensive computational experiments by using different problem instances that rank reversal problems can be avoided in WASPAS when modified Max and Max–Min normalization techniques are used.
