Content uploaded by Yan Yang
Author content
All content in this area was uploaded by Yan Yang on Jul 21, 2018
Content may be subject to copyright.
Intl. Trans. in Op. Res. 00 (2018) 1–26
DOI: 10.1111/itor.12569
INTERNATIONAL
TRANSACTIONS
IN OPERATIONAL
RESEARCH
New doctors ranking system based on VIKOR method
Junhua Hu, Xiaohong Zhang, Yan Yang, Yongmei Liu and Xiaohong Chen
School of Business, Central South University, Lu Shan Nan Lu 932, Yue Lu District, Hunan Province,
410083 Changsha, P.R. China
E-mail: hujunhua@csu.edu.cn [Hu]; 161611125@csu.edu.cn [Zhang]; gabbylovie@csu.edu.cn [Yang];
liuyongmeicn@163.com [Liu]; cxh_csu@csu.edu.cn [Chen]
Received 19 January 2018; received in revised form 24 May 2018; accepted 30 May 2018
Abstract
Nowadays, we can use different websites that help us make decisions about various aspects of our lives.
However, privacy protection prevents websites from providing personalised guidelines to users. We propose
a novel doctor-ranking system (DRS) based on multi-criteria group decision-making (MCGDM) method
to address the problems of privacy protection. The following aspects differentiate our proposed DRS model
from previous works: (a) textual information reviews are used to identify user preferences and complementary
criteria, (b) criteria weights are determined by term frequency inverse document frequency (TF-IDF) instead
of Delphi method or expert opinion, (c) intuitionistic fuzzy sets (IFSs) are used to replace sentiment analysis
to express subjective user criteria, and (d) VIsekriterijumsko KOmpromisno Rangiranjie (VIKOR) method for
MCGDM with IFSs is used to solve the doctor-ranking problem. We apply our proposed model to datasets
from Haodf.com to compare the performance of our method with that of sentiment analysis and technique
for order performance by similarity to ideal solution (TOPSIS) methods. The experimental results show that
our method provides accurate ranking and increases the reliability of DRS.
Keywords: reviews of textual information; TF-IDF; VIKOR method; doctors ranking
1. Introduction
Information technology has changed the way Internet users make decisions. Crucial professional
knowledge from various websites can be accessed at any time to assist the decision-making process
of users around the world (Ardissono et al., 2003). For example, users can select different books
on the basis of ratings on Amazon and other users’ book reviews. Likewise, health-related review
websites, such as Vital.com, Healthgrades.com and Yelp.com, have changed the way patients select
medical service. Hoens et al. (2010) found that patients prefer finding appropriate doctors through
websites. Through these sites, users can obtain detailed information about doctors and communicate
with them conveniently. Patients also can customise their search preferences for matches. In general,
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Published by John Wiley & Sons Ltd, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main St, Malden, MA02148,
USA.
2Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
a patient selects doctors on the basis of their disease or the doctor’s specialisation (Chen, 2016).
For example, a patient living in Jiangsu, China, can find the best doctors across the country via
Haodf.com, a health-related website. Nevertheless, medical information overload makes finding an
appropriate doctor difficult. Therefore, websites should provide accurate doctor rankings to users
(Hu et al., 2017).
Online doctor selection is more convenient than traditional doctor selection. Therefore, it should
provide professional medical information to users. However, the provision of accurate medical
information is challenged by the following concerns.
1.1. Privacy protection
Doctor-ranking websites protect patient privacy by concealing individual information, including
name, ID number, age and gender. Although privacy protection allows users to express themselves
safely and freely, it also hinders the analysis of user behaviour. Thus, these websites cannot provide
personalised recommendations to users. In this case, the doctor-ranking system (DRS) runs under
the premise of privacy protection. The user is treated as an independent individual, and his comments
are not directly related with DRS.
1.2. Lack of criteria
The development of DRS is slower than that of other recommendation systems because website
development is limited by the lack of evaluation criteria. Moreover, Hao (2015) pointed that the
development of DRS in China is slower than that in other developed countries. Hao and Zhang
(2016) stated that the most popularly reviewed topics on health-related websites are the experiences
of finding doctors, medical skills and bedside manner. Haodf.com, however, only has three criteria:
bedside manner, cost and recovery condition. These criteria cannot fully represent patient prefer-
ence. Furthermore, a survey indicated that the majority of users on Haodf.com provided positive
evaluations of treatment effect and bedside manner, but the textual information is often inconsistent
with star-rating (Hao, 2015). Thus, the system for evaluating textual information should be revised.
1.3. Professionalised demand
A DRS should provide desirable experiences to users (Guan et al., 2009). However, users may
search for medical service by disease or region and may obtain high numbers of irrelevant results.
For example, some users may prefer doctors who provide services within their vicinity, and others
may prefer out-clinic treatment. Unfortunately, such specialised user needs cannot be satisfied at
present (Schuckert et al., 2015). The majority of existing DRSs rank doctors on the basis of patient
evaluation and only provide basic information (e.g. profession, title, hospital). Therefore, researchers
in the DRS field have begun to focus on constructing a decision model to assist patients in finding
the appropriate doctor (Gong and Sun, 2011; Sun et al., 2017b) and to assist doctors in providing
the correct diagnosis (De et al., 2001). For example, Hoens et al. (2010) used patient evaluations to
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 3
Fig. 1. A screenshot from Haodf.com. [Colour figure can be viewed at wileyonlinelibrary.com]
develop a privacy-friendly recommendation framework for physicians. User reviews have been used
to revise original scores to generate accurate ratings (Schuckert et al., 2015; W´
ojtowicz et al., 2016).
Nevertheless, these models do not simultaneously consider incomplete information and textual
information reviews. Furthermore, they cannot solve multi-criteria decision problems.
1.4. Reviews of textual information
User reviews have an important influence on ratings. Textual information reviews reflect attitudes
towards product selection and affect product ratings (Martellos, 2012). They can compensate for
the shortage of criteria and better represent patient preferences. For example, as shown in Fig. 1, if
a patient wants to use Haodf.com to find a specific doctor who specialises in coronary heart disease,
the website will present basic doctor information, such as specialisation, location and numerical
ratings. Irrelevant information complicates the immediate selection of a specific doctor. Therefore,
the website needs to provide users with professional rankings.
To deal with the above challenges, we propose a DRS with high accuracy. Our proposed DRS
integrates intuitionistic fuzzy sets (IFSs), term frequency inverse document frequency (TF-IDF)
and VIsekriterijumsko KOmpromisno Rangiranjie (VIKOR) methods with multi-criteria group
decision making (MCGDM) under incomplete information (Wang and Zhang, 2013; Meng and
Chen, 2015; Yu et al., 2018). It is remarkable that VIKOR method has quicker analysing ability, less
memory, and better accuracy than other methods (Asgharizadeh et al., 2017). Patients can search
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
4Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
for the appropriate doctor on the basis of their needs. The proposed model can be used with different
search criteria. For example, users may select doctors by specialisation. Our proposed DRS can
provide accurate evaluation-based doctor ranking. We introduce related works on DRS below.
Recommendation systems have been extensively investigated. These systems can help users effec-
tively identify their needs. One method for improving recommendation accuracy is to extract per-
sonalised information by comparing products that users purchase (Schafer et al., 2001; Manikrao
and Prabhakar, 2005; Chen et al., 2008; Hwang, 2018). Another method is to transform the user’s
subjective perceptions into the latent quality of service and products (Weng and Liu, 2004; Cao and
Li, 2007). Cao and Li (2007) focused on the special features of products to provide recommenda-
tions. Natural language processing techniques based on social network analysis are used to provide
the star rating, which is based on previous experiences of other users (Dev and Anjum, 2016; Sobrie
et al., 2018). Combining text mining with decision methods improves recommendation accuracy
(Huang et al., 2012; Chen and Chen, 2014, 2015; Yu et al., 2017; Zhang et al., 2017; Wang et al.,
2018b). However, these recommendation systems are unsuitable for doctor selection. User demands
are uncertain given privacy concerns and insufficient criteria. Thus, professional recommendations
are not reproducible. Zhang et al. (2016) used sentiment analysis and topic model to discover user
preferences and doctor feature distribution from Yelp.com, which provides personal information
(rating, history evaluation and social networking). They also combined matrix factorisation method
with sentiment analysis to provide accurate medical recommendation.
DRS also makes use of IFSs to describe the importance of criteria. IFSs, which was introduced by
Atanassov (1994), is characterised by a membership and non-membership function (Zadeh, 1965).
IFS provides the possibility of measuring uncertainty and hesitation (Liu and Wang, 2007; Gao
et al., 2018b; Wei and Gao, 2018). Over the past few decades, this theory has coalesced into a stable
subject and has been applied in various fields, such as decision making (Yager, 2004; Garg, 2016;
Hu et al., 2018; Gao et al., 2018a; Wei, 2018a; Wei and Lu, 2018), medical diagnosis (De et al.,
2001; Rajarajeswari and Uma, 2014; Sun et al., 2017a; Wei and Wei, 2018) and pattern recognition
(Meng and Chen, 2016; Wei, 2018b). Compared with other fuzzy sets, IFSs can be easily applied
to DRS. In this study, doctor ranking is dependent on the judgment and intuition of patients.
Therefore, MCGDM with IFSs is useful for measuring uncertainty in DRS. MCGDM also has
been studied by many researchers. The VIKOR (Roostaee et al., 2012) and TOPSIS (Technique
for order performance by similarity to ideal solution) equation methods are applied in MCGDM
to select optimal schemes (Boran et al., 2009). Both methods are appropriate for computing the
closeness of big data to the ideal and other alternatives to determine ranking. The weights of
decision makers (DMs) and criteria play an important role in the group decision-making process
of weighted aggregation (Wang et al., 2018a). The majority of methods directly assigns weights to
a real number or a linguistic term (Wei, 2010; Dammak et al., 2015). These weights, however, are
determined by a new method in this paper. In the presented method, we use TF-IDF to obtain the
weights of criteria and similarity measure to determine the weights of DMs. TF-IDF (Jones, 1972) is
used to deal with textual information. TF-IDF has been further developed and improved by experts
(Martineau and Finin, 2009; Paltoglou and Thelwall, 2010; Paik, 2013). Given that DRS relies on
reviews of textual information, TF-IDF is more suitable for determining criteria weights than other
methods, e.g. Delphi method, analytic hierarchy process (AHP) or principal components analysis
(PCA). The similarity measure is the common method that has been applied to supplier selection
(Roostaee et al., 2012).
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 5
Fig. 2. The framework of ranking.
This study provides the following contributions: (a) the use of textual information reviews to
discover patient preference, (b) the use of TF-IDF instead of Delphi method or expert opinion to
determine criteria weights, (c) the replacement of sentiment analysis with IFSs to express the user’s
subjective perceptions and (d) the use of VIKOR method with IFSs for MCGDM to solve the
doctor-ranking problem in an incomplete and uncertain information environment.
The rest of the paper is organised as follows. Related theories about DRS, including IFS, TF-IDF
and VIKOR methods, are introduced in Section 2. The steps of model construction, including data
preparation, transformation model and ranking, are discussed in Section 3. Doctor ranking based
on the model discussed in Section 3 is discussed in Section 4. We compare the VIKOR method with
other methods to verify its performance. The ranking process is shown in Fig. 2.
2. Preliminaries
2.1. IFSs
Definition 1 (Atanassov, 1986). Let X ={x1,x2,...,xn}be a finite set, and an IFS A in X has the
following form:
A={x,μ
A(x), vA(x)|x∈X},
μA(x)∈[0,1],vA(x)∈[0,1],
where μA(x)and vA(x)are the respective membership and non-membership degrees of each element
x∈X that satisfy the condition 0≤μA(x)+vA(x)≤1.Let πA(x)=1−μA(x)−vA(x). Thus,
0≤πA(x)≤1,πA(x)is the hesitation degree.
Definition 2 (Atanassov, 1986; Burillo and Bustince, 1995). Let A and B be two IFSs in the set X .
Then,
(1) A +B={x,μ
A(x)+μB(x)−μA(x)μB(x), νA(x)vB(x)|x∈X}.
(2) AB ={x,μ
A(x)μB(x), νA(x)+vB(x)−νA(x)vB(x)|x∈X}.
(3) λA={x,1−(1−μA(x))λ,(vA(x))λ|x∈X}.
(4) Aλ={x,(μ
A(x))λ,(1−vA(x))λ|x=X}.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
6Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
Corollary 1 (Xu, 2007). Let {A1,A2,...,An}be the sets with n intuitionistic fuzzy (IF) numbers, where
Aj=(uj,vj), j=1,2,...,n. The weighted arithmetic average is defined as follows:
n
j=1
εjAj=⎛
⎝1−
n
j=1
(1−uj)εj,
n
j=1
(vj)εj⎞
⎠,(1)
where εjis the weight of A jand j =1,2,...,n,0≤εj≤1,n
j=1εj=1.
2.2. Ranking method for IFSs
Here, we introduce the basic-ranking method, which is based on the score and accuracy function.
Definition 3. Let A =(μA,vA)be an IF number (IFN). Xu (2007) defines a score function s as a
measurement of IFN A as follows:
s(A)=μA−vA.Thus,−1≤s(A)≤1.
Definition 4 (Hong and Choi, 2000). Let A =(μA,vA)be an IFN. The accuracy function σof A can
be defined as follows:
σ(A)=μA+vA,Likewise, 0≤σ(A)≤1, and σ(A)=μA+vA=1−πAis easily inferred.
Thus, the accuracy function represents the non-hesitant degree, and the score function and accu-
racy function are used to rank IFS. In general, the score function is more crucial than the accuracy
function.
Definition 5. Based on score function s and accuracy function σ, Xu (2007) defined an order relation
between IFNs A1=(μ1,ν
1)and A2=(μ2,v2)as follows:
(1) if s(A1)>s(A2),thenA
1is larger than A2.
(2) If s(A1)=s(A2),
(2a) And if σ(A1)=σ(A2),thenA
1is equal to A2.
(2b) And if σ(A1)≺σ(A2),thenA
1is smaller than A2.
(2c) And if σ(A1)σ(A2),thenA
1is larger than A2.
Definition 6 (Roostaee et al., 2012). To define the distance between IFSs A and B, a normalised
Hamming distance D is introduced.
D(A,B)=1
2μA−μB−νA−vB.(2)
Definition 7. Defined as a metric, let X1=(x1
ij), X2=(x2
ij), i=1,2,...,m,j=1,2,...,nbetwo
matrices, where xt
ij =(μt
ij,vt
ij)are also IFNs. The distance measurement between two matrices based
on Hamming distance DHis defined as follows (Roostaee et al., 2012):
DHX1,X2=1
2mn
m
i=1
n
j=1μ1
ij −μ2
ij+v1
ij −v2
ij.(3)
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 7
Boran et al. (2009) defined the normalised Euclidean distance DEas follows:
DEX1,X2=1
2mn
m
i=1
n
j=1μ1
ij −μ2
ij2
+v1
ij −v2
ij2
.(4)
2.3. Model of feature weight
Term frequency (TF) refers to the frequency at which a given word appears in the file, and inverse
document frequency (IDF) is the measure of the importance of the term (Wu et al., 2008). Let tf
represent TF and idf represent IDF.
tfi,j=count(ti,dj),
idfi=log |D|
j:ti∈dj
,
where tidenotes the ith term, i=1,...l,djis the jth document, j=1,...n,|D|is the total number
of documents and |{ j:ti∈dj}| is the number of documents containing ti.However,thedivisor
becomes zero if the word is not in the document body. Thus, |{ j:ti∈dj}| are usually represented as
1+|{j:ti∈dj}|.To perform word sense disambiguation for a target word w, the most likely sense
of wmay appear in a long document. Classical TF-IDF is expressed as follows:
TF-IDF =tfi,j×idfi.
The improvement in document length has an important role in effective-term weighting (Paik,
2013). We can easily infer that the number of terms is not distinct. Given that we tend to prefer long
documents, the improved term frequency (ITF) is as follows:
ITF(t,d)=RITF(t,d)
1+RITF(t,d),
RITF(t,d)=⎧
⎪
⎪
⎨
⎪
⎪
⎩
0,Avg .TF(t,d)=0
log2(1+TF(t,d))
log2(1+Avg.TF(t,d)),Av g .TF(t,d)= 0
,(5)
where TF(t,d)and Avg.TF(t,d)denote the frequency of the term tin dand the average TF of d.
TF aims to assign higher weights to documents that contain more terms than to documents that
contain fewer terms. IDF essentially quantifies the above intuition. We hypothesise that if two terms
have same document frequency, then term discrimination also should increase with TF. So, the final
term discrimination TDF is computed as Equation (6):
TDF(t,d)=IDF(t,d)×AEF(t,d)
1+AEF(t,d),(6)
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
8Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
where AEF(t,d)=CTF(t,c)
|D|and CTF(t,c)are the total occurrence of the term tin the entire
collection. Therefore, the weight of DM to feature w(i,j)is shown as follows:
w(i,j)=RITF(t,d)×TDF(t,d).
Finally, normalise the weight obtained by computing, w(i,j). Then the final weight of features
wjis shown as Equation (7):
wj=l
i=1wi,j
l
i=1n
j=1wi,j
.(7)
2.4. VIKOR method
Roostaee et al. (2012) extended VIKOR method in intuitionistic fuzzy environment, and applied it
into supplier selection. Here, because most of patients are risk-reversion, we utilise this method to
select the appropriate doctor among malternatives A={A1,A2,...,Am}on the assumption that k
DMs D={D1,D2,...,Dk}exist. Some useful steps are introduced as follows:
Step 1. Construct the normalised intuitionistic matrix.
Relatively important criteria Cjof alternatives Aiare derived from reviews. Let
C={C1,C2,...,Cn}be the set of criteria and Dt(t=1,...,k)represent each DM, nis the number
of criteria, mis the number of alternatives and kis the number of DMs, where 1 ≤j≤n,1≤i≤m
and 1 ≤t≤k.
All individual evaluation should be considered in MCGDM problem by transforming individual
opinions into the group-decision matrix, which is obtained in accordance with Definition 2. Let X
denote the aggregation matrix, where X=k
t=1ωtXtand X=(xij)m×n=(μij,vij)m×n
xij =1−
k
t=1
(1−μt
ij)ωt,
k
t=1
(vt
ij)ωt.(8)
Step 2. Determine DMs weight.
To improve decision making, we need to assign weights to DMs in accordance with importance.
First, we obtain the ideal matrix X∗=(x∗
ij)m×n=(μ∗
ij,v∗
ij)m×nbased on Equation (8), where
ωt=1
k. Specially, ideal matrix is the particular form of normalised intuitionistic matrix.
Then, DMs weight should be satisfied: a short distance between the decision of Dand the ideal
decision indicates high Dweight. Therefore, the similarity measure between Dand the ideal decision
matrix is defined as follows:
S(Xt,X∗)=SXt,Xe∗
S(Xt,X∗)+S(Xt,Xe∗),
Xe∗=(x∗
ij)=(ν∗
ij,μ
∗
ij)
(9)
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 9
where, S(X1,X2)=1−D(X1,X2),Xe∗is the negative ideal point, D(X1,X2)is the distance
between X1and X2and D(X1,X2)is obtained with Equation (3).
Finally, assuming that ωtrepresents the weight of Dt,eachωt(t=1,2,...,k)is defined based on
similarity measurement as shown below:
ωt=S(Xt,X∗)
k
i=1S(Xi,X∗)
,(10)
clearly, ωt≥0andk
t=1ωt=1.
Step 3. Determine criteria weights.
Criteria weights are denoted by W=(w1,w2,...,wn),wherewnrepresents the importance of
criterion Cj. The weights are expressed by IFSs in the conventional VIKOR method. This method
requires DMs to evaluate criteria. However, websites do not provide evaluation values. In this study,
we utilise the TF-IDF method to determine criteria weights. TF-IDF is derived from text mining.
The specific practice is given in Section 2.3.
Step 4. Determine the best and the worst values.
Let Q+and Q−separately be the best and the worst value for all values, respectively. These values
are defined as follows:
Q+
j=max
i(Qij), i=1,2,...,m,(11)
Q−
j=min
i(Qij), i=1,2,...,m.(12)
Thus, the maximum or minimum element is determined.
Step 5. Compute the separation measure.
Let Sidenote the separation measure. The best value S+
iand the worst value S−
iare computed as
follows:
S+
i=
n
j=1
DQ+
j,xij
DQ+
j,Q−
jwj,i=1,2,...,m,(13)
S−
i=max
j⎛
⎝
DQ+
j,xij
DQ+
j,Q−
j⎞
⎠wj,i=1,2,...,m,(14)
S+
iand S−
iare IFNs.
Step 6. Compute the values of fi.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
10 Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
The value fi(i=1,2,...,m)is defined as follows:
fi=vDS+
i,S∗
D(S+,S∗)+(1−v)DS−
i,S−∗
D(S−,S−∗),(15)
where S∗=miniS+
i;S+=maxiS+
i,S−∗=miniR+
i;S−=maxiR+
iand 0 ≤v≤1, vis the weight
for the strategy of maximum group utility. Accordingly, 1 −vis the weight of individual regret.
v>0.5 indicates that S+
iis more important than S−
i. Otherwise, v<0.5 represents that S−
iis rela-
tively important. The weight of vinfluences the ranking of alternatives, and it is always determined
by experts. Finally, we conclude the ranking in accordance with Steps 1–6.
3. Doctor-ranking model
Users tend to seek a doctor who has excellent medical skills and good service attitude. We compre-
hensively employed data preparation, transformation and ranking modules to construct an effective
ranking module. The objectives of the three modules are described as follows:
(1) Data preparation includes data collection, feature extraction and matching on the basis of
textual information review.
(2) Transformation converts feature values into numerical values.
(3) Ranking computes doctor rankings in accordance with user evaluation.
The proposed system can offer a list of doctors to users. The list is commonly determined on the
basis of doctor evaluation. The entire ranking process is discussed in the following subsections.
DRS contains three problems. First, privacy protection prevents patients from disclosing infor-
mation and websites from providing personalised professional service to patients. In this study,
we rank doctors on the basis of patient demands. IFSs are used to replace original data given the
vagueness of user evaluation. Moreover, we treat every patient as an independent DM and transform
the ranking problem into an MCGDM problem. In this way, we can rank doctors on the premise
of privacy protection. Second, the shortage of criteria is another challenge. Summarising criteria
emphasised by patients is difficult, and obtaining criteria through experience is subjective. Thus,
we exploit textual information to discover patient preference to identify relevant criteria. Lastly,
we rank doctors on the basis of evaluation and improve system accuracy by using evaluations that
include linguistic and textual information.
3.1. Data pre-processing
Data pre-processing aims to transform original data into numerical values. This section provides a
discussion of three modules: Data preparation, feature extraction and transformation.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 11
Fig. 3. The procedure of collecting data. [Colour figure can be viewed at wileyonlinelibrary.com]
3.1.1. Data collection
We collected experimental data from Haodf.com during April 2017. Octopus collector software
was used to collect evaluations of the top 12 doctors. Figure 3 shows the specific data collection
procedure. For every doctor, we concentrated on 12 criteria: Doctor’s name, patient’s name, medical
purpose, treatment, medical effect, bedside manner, textual information reviews, reason for choosing
the doctor, method for setting up an appointment, recovery condition, medical fee and disease.
Figure 4 is an example of 12 criteria for Dr. Liu given by Mr. Li on Haodf.com. The form of original
data we collected is shown in Fig. 5. We then processed textual information data.
3.1.2. Feature extraction
After data collection, we performed Chinese word segmentation with Rstudio (Jiebar). This step
was performed to remove stop words. We used a database of stop words from experts (Hao and
Hao, 2008). For example, most patients would say thank,thanks and doctor as calls in their textual
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
12 Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
Fig. 4. An example of 12 criteria to Dr. Liu given by Li on Haodf.com. [Colour figure can be viewed at
wileyonlinelibrary.com]
Fig. 5. A part of records about Dr. Liu.
information. We removed these words from the lists because they cannot represent any opinion.
Moreover, Chinese is different from English because it does not have specific plural terms and verb
tenses. Then, we used Rstudio to automatically obtain the distribution of high-frequency words
based on TF-IDF. We selected the top 1000 high-frequency words to extract doctor characteristics.
Finally, we divided high-frequency words into each criterion in accordance with a standard (Stern
et al., 2009). Figure 6 shows the users’ criteria for doctors.
After classification, we analysed the evaluation grades of criteria. Two types of features are only
considered: one is single adjective, the other is bi-gram. The single adjective was used to extract
particular features. For example, approachable was only used to describe attitude. We used bi-gram to
extract features (e.g. skill or personality) from vague adjectives (such as good or very good). Rstudio
was used to segment these adjectives from textual information reviews. These adjectives facilitated
representing and calculating evaluation grades of criteria because they are more accurate than
subjective analysis in mining preferences. The adjectives extracted from the textual information
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 13
Fig. 6. Domains of global essential requirements. [Colour figure can be viewed at wileyonlinelibrary.com]
Table 1
The adjectives included in patient reviews
Reviews Adjec ti ve s
I was the family of 8th ward at three bed. Patient had
lung condition a few months ago, then we went to a
local hospital, but the local medical level is
relatively limited, so we find Dr. Liu from
Haodf.com, he is approachable with excellent
medical skills, and they arrange bed for us
immediately. After examination, the condition is
serious with 80% blood vessel blocking . . . . . .
After nine months we accept examination again,
the condition is getting better and better, thank
Dr. Liu very much.
Excellent, approachable, serious, getting better
of one user’s review for one doctor is shown in Table 1. Classification and segmentation results
are shown in Fig. 7. Classification adds three criteria: medical skill, medical ethics and treatment
promptness. The remaining criteria were provided by Haodf.com. Each criterion contains adjectives
that represent evaluation grades.
3.1.3. Transformation module
We transformed adjectives and linguistic terms into IFSs to aggregate patient evaluation. Adjectives
are easily transformed into IFSs in accordance with five evaluation grades or nine grades as needed.
Adjective transformation has to address two problems: one is the matching problem between high-
frequency words and textual information, and the other is sparse data.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
14 Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
Fig. 7. The consequence of classification.
Table 2
The matching problem of recovery condition
Evaluation Total recovery (TR) Improved (I) Unchanged (U) Exacerbated (E)
Total recov e r y ( T R ) TR I U E
Improved (I) I I U E
Unchanged (U) U U U E
Exacerbated (E) E E E E
The matching problem arises between high-frequency words and textual information because the
original criteria of bedside manner and recovery condition on Haodf.com may be presented twice.
We consider the worse evaluation if conflicting evaluations for the criterion of recovery exists. If
a patient evaluates doctor more than once, we also select the worse evaluation. Table 2 shows the
matching problem of bedside manner. The textual information review contains too many adjectives
for the criterion of bedside manner. For example, the terms patient,serious,careful and relatable
are used to describe the doctor’s attitude. However, these terms are vague and have no basis for
comparison. Thus, we adopted the values that are provided by websites. The textual information
reviews are only used to determine criteria weights. The adjectives of the new criteria medical skill,
medical ethics and prompt treatment are easily transformed into IFSs. As shown in Fig. 8, we use
nine grades to transform medical skill given the negligible differences in adjectives for this criterion.
Medical ethics and prompt treatment can be easily recognised by five grades in accordance with
Tables 3a and 3b.
Sparse values are considered criteria without patient comments. In this case, we always consider
that patients have neutral attitudes towards doctors. Thus, we adopted the term fair (F) to replace
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 15
Fig. 8. Transformation process between medical skill and nine grades. [Colour figure can be viewed at
wileyonlinelibrary.com]
Table 3 a
Linguistic terms for ranking alternatives
Linguistic terms IFNs
Very important (VI) 1.00,0.00
Important (I) 0.80,0.10
Rather important (RI) 0.60,0.00
Medium (M) 0.00,0.50
Totally important (TI) 0.00,1.00
I do not know 0.50,0.50
Table 3 b
The transformation between linguistic terms and IFSs
Linguistic terms IFNs
Extremely good (EG) 1.0,0.0
Very very good (VVG) 0.90,0.10
Very good (VG) 0.80,0.10
Good (G) 0.70,0.20
Medium good (MG) 0.60,0.30
Fair (F) 0.50,0.40
Medium bad (MB) 0.40,0.50
Bad (B) 0.25,0.60
Ver y b a d ( VB) 0.10,0.75
Very very bad (VVB) 0.10,0.90
sparse data, unless all criterion values are missing. We deleted data if all criterion values were
missing. Table 4 shows the specific conditions.
3.2. Ranking module
To address the problem of posed by evaluations from different patients for the same doctor, we
used the MCGDM approach to rank doctors. In our model, we considered that every patient is an
independent DM and then resolved the ranking problem through VIKOR. We describe the specific
process below:
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
16 Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
Table 4
The sparse data condition included in evaluation
Number of data Type of problem for data The method to deal with data
A few values of criteria Missing Fill data with fair term
All values of criteria Missing Delete this piece of data
Step 1. After obtaining the results of word segmentation, calculate the frequency of key words
and categorise every high-frequency word as shown in Fig. 7. Then, all frequencies are added in
accordance with categories. As shown in Table 2, addition is performed only once for the same
property with different degrees. Finally, count the length of every evaluation and frequency of
documents.
Step 2. Calculate feature weights based on Equations (5)–(7).
Step 3. Determine weights of patients. First, complement sparse data in accordance with Table
4. Second, construct the ideal matrix on the basis of Equation (8). Lastly, calculate the distance
between patient evaluation and ideal matrix on the basis of Equations (9) and (10).
Step 4. Construct the normalised matrix on the basis of Equation (8).
Step 5. Determine criteria weights in accordance with Equation (7). The TF-IDF method is utilised
to determine criteria weights, and the specific practice is given in Section 2.3.
Step 6. Determine the best Q+
jand the worst Q−
jvalues on the basis of Equations (11) and (12).
Step 7. Compute the separation measure S+
iand S−
ion the basis of Equations (13) and (14).
Step 8. Compute the values of fion the basis of Equation (15).
Step 9. Compare final ranking with the ranking provided by TOPSIS and sentiment analysis method.
The TOPSIS method is different from VIKOR method in Steps 6–8 and is easier than the VIKOR
method. Rstudio is used to achieve sentiment analysis.
4. Case study: Haodf.com
Haodf.com ranks thousands of doctors on the basis of disease or specialisation. Here, we
use coronary heart disease as an example. We selected the top 12 doctors across China.
Let Ddenote the set of doctors and the original rating of D(D1,D2,...,D12)Tis denoted
as RD=(5.0,4.4,4.4,4.3,4.3,4.3,4.2,4.2,4.2,4.2,4.2,4.1)T. Two thousand three hundred and
forty patients evaluated the selected doctors. The number of patients of the 12 doctors is
(1179,145,142,168,122,118,79,102,80,85,75,45)T. First, we take distance into consideration.
We found out that patients who reside in different areas may seek treatment from renowned doctors
in different localities. Moreover, despite the reputation of hospitals, patients opt to receive treat-
ment nearby or all over China. Therefore, we ignored the criteria of distance. The criteria of bedside
manner, treatment effect and state of illness are given on the website, and other criteria were derived
from textual information reviews.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 17
Table 5
The results of frequency
Keys Frequency
Great 88
Primary 34
Highly 28
Nice 32
Fas t 13
... ... ... ...
Effective 5
Anxious 2
Emergency 4
Embarrassed 2
Table 6
TF of key words
Frequency Percent Valid percent
Valid State of illness 518 9.6 9.6
Very fast 2 0.0 0.0
Very patient 11 0.2 0.2
Beside manner 81 1.5 1.5
Excellent 321 5.9 5.9
Very excellent 469 8.7 8.7
Recovery 55 1.0 1.0
successful 21 0.4 0.4
... ... ... ... ... ... ... ...
Treatment effect 32 0.6 0.6
4.1. The ranking model with the VIKOR method
Step 1. Extract attributes from textual information.
Rstudio was used to separate key words from textual information. In this step, we first selected
1000 key words and then extracted useful terms from those words. Table 5 shows the TF of reviews.
Then, these terms were classified into five categories after stop words were reviewed. Finally, as
shown in Table 6 and Fig. 9, five criteria were used to rank doctors.
Step 2. Calculate the frequency of every word. This step aims to obtain TF and document frequency.
Then, we used Equations (5)–(7) to obtain the weights of attributes. Table 7 does not consider the
conventional values of text length, whereas Table 8 considers text length. Finally, we obtained
weights on the basis of classification. The weights of each criterion were calculated on the basis of
Equation (7).
wj=(0.234,0.188,0.270,0.153,0.155)T.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
18 Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
Fig. 9. The results of classification. [Colour figure can be viewed at wileyonlinelibrary.com]
Table 7
The conventional TF-IDF of textual information
Patient Medical skill Medical ethics Beside manner Speed of remedy Recovery
Text 1 0.139 0.000 0.241 0.243 0.245
Text 2 0.000 0.000 0.498 0.502 0.000
Text 3 0.361 0.000 0.000 0.000 0.639
Text 4 0.122 0.176 0.211 0.213 0.000
Text 5 0.511 0.000 0.000 0.000 0.000
Text 6 0.000 1.000 0.000 0.000 0.000
Text 7 0.000 0.000 0.000 0.000 0.000
Text 8 0.294 0.425 0.000 0.000 0.000
Text 9 0.212 0.306 0.000 0.000 0.000
... ... ... ... ... ... ... ... ... ... ... ...
Table 8
Considering the length of documents
Patient Medical skill Medical ethics Beside manner Speed of remedy Recovery
Text 1 0.135 0.000 0.139 0.299 0.303
Text 2 0.000 0.000 0.305 0.695 0.000
Text 3 0.308 0.000 0.000 0.000 0.692
Text 4 0.118 0.206 0.123 0.262 0.000
Text 5 0.521 0.000 0.000 0.000 0.000
Text 6 0.000 1.000 0.000 0.000 0.000
Text 7 0.000 0.000 0.000 0.000 0.000
Text 8 0.273 0.476 0.000 0.000 0.000
Text 9 0.192 0.334 0.000 0.000 0.000
... ... ... ... ... ... ... ... ... ... ... ...
Sum 131.472 105.951 152.138 85.924 87.357
Step 3. Determine the weight of each patient.
First, we converted linguistic information and adjectives (shown in Table 1) into IFNs. The
specific transformation rules are shown in Tables 2 and 3. In this step, we complemented sparse data
in accordance with Table 4. After transformation, we constructed the ideal matrix to determine the
weights of patients (see Table 9). The matrix is shown in Table 10.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 19
Table 9
The weights of patients
Patients P1 P2 P3 . . . . . . P2338 P2339 P2340
ω0.0007 0.0008 0.0008 . . . . . . 0.0216 0.0216 0.0218
Table 1 0
The ideal matrix
Doctors Medical skill Beside manner Speed of remedy Recovery
D1 (0.716,0.210) (0.765,0.128) (0.548,0.346) (0.530,0.366)
D2 (0.787,0.134) (0.784,0.113) (0.517,0.381) (0.511,0.388)
D3 (0.819,0.103) (0.779,0.117) (0.514,0.385) (0.522,0.376)
D4 (0.808,0.109) (0.774,0.120) (0.507,0.392) (0.549,0.345)
D5 (0.967,0.011) (0.952,0.013) (0.763,0.148) (0.784,0.130)
D6 (0.822,0.101) (0.777,0.118) (0.521,0.377) (0.512,0.386)
D7 (0.823,0.102) (0.775,0.120) (0.518,0.379) (0.544,0.351)
D8 (0.822,0.104) (0.777,0.118) (0.517,0.381) (0.524,0.373)
D9 (0.810,0.107) (0.788,0.110) (0.506,0.393) (0.514,0.385)
D10 (0.807,0.102) (0.790,0.108) (0.530,0.366) (0.539,0.359)
D11 (0.829,0.100) (0.769,0.125) (0.533,0.361) (0.573,0.318)
D12 (0.825,0.103) (0.780,0.116) (0.506,0.394) (0.522,0.376)
Table 1 1
The best and worst values
Criteria Medical skill Medical ethics Beside manner Speed of remedy Recovery
Qj
+(0.830,0.100) (0.785,0.112) (0.788,0.109) (0.547,0.347) (0.567,0.325)
Qj
−(0.708,0.216) (0.758,0.134) (0.759,0.133) (0.505,0.394) (0.510,0.389)
Then, we calculated the final weights of each patient. The result is shown in Table 9. The result
indicated that the weights of patients who evaluated the same doctor are not different. We can
conclude that all patients provided rational doctor evaluations.
Step 4. Determine the best and the worst values for doctors.
The best and the worst values for doctors are shown in Table 11.
Step 5. Construct the aggregation matrix.
The weights of patients were used to aggregate the final matrix in accordance with Equation (8).
Table 12 shows the aggregation results.
Step 6. Calculate the separation measure.
The values were calculated for all doctors and are shown in Table 13.
Step 7. Compute the values of fi.
Table 14 shows the computation of various values.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
20 Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
Table 1 2
The aggregation matrix
Doctors Medical skill Medical ethics Beside manner Speed of remedy Recovery condition
D1 (0.708,0.216) (0.760,0.132) (0.759,0.133) (0.547,0.347) (0.530,0.367)
D2 (0.782,0.138) (0.777,0.119) (0.781,0.115) (0.517,0.381) (0.510,0.389)
D3 (0.818,0.103) (0.770,0.125) (0.775,0.120) (0.513,0.385) (0.521,0.377)
D4 (0.807,0.110) (0.758,0.134) (0.770,0.124) (0.506,0.392) (0.546,0.348)
D5 (0.830,0.100) (0.784,0.112) (0.782,0.114) (0.517,0.381) (0.519,0.379)
D6 (0.821,0.102) (0.772,0.122) (0.772,0.122) (0.521,0.377) (0.512,0.386)
D7 (0.822,0.102) (0.772,0.122) (0.771,0.123) (0.519,0.379) (0.543,0.352)
D8 (0.821,0.105) (0.772,0.123) (0.773,0.121) (0.516,0.382) (0.524,0.373)
D9 (0.808,0.108) (0.783,0.114) (0.786,0.111) (0.507,0.393) (0.514,0.385)
D10 (0.807,0.102) (0.785,0.112) (0.788,0.109) (0.528,0.368) (0.537,0.361)
D11 (0.829,0.100) (0.768,0.126) (0.763,0.129) (0.530,0.365) (0.567,0.325)
D12 (0.825,0.103) (0.773,0.122) (0.776,0.119) (0.505,0.394) (0.522,0.377)
Table 1 3
The values of Sand Rfor all doctors
D1 D2 D3 D4 D5 D6 D7 D8 D9 D10 D11 D12
S0.059 0.035 0.046 0.057 0.022 0.049 0.043 0.040 0.034 0.030 0.039 0.042
R0.029 0.009 0.013 0.018 0.008 0.017 0.017 0.015 0.013 0.020 0.023 0.013
Table 1 4
The values of fi
v=1v=0.5v=0.5v=0.25 v=0
f10.000 0.000 0.000 0.000 0.000
f20.655 0.729 0.803 0.878 0.952
f30.353 0.454 0.556 0.657 0.758
f40.051 0.170 0.289 0.409 0.528
f51.000 1.000 1.000 1.000 1.000
f60.284 0.359 0.434 0.509 0.584
f70.438 0.477 0.515 0.554 0.592
f80.522 0.556 0.590 0.624 0.658
f90.689 0.710 0.731 0.752 0.773
f10 0.786 0.696 0.606 0.516 0.426
f11 0.553 0.482 0.411 0.339 0.268
f12 0.467 0.543 0.618 0.693 0.768
Step 8. Rank the doctors.
We utilised the above steps to obtain the list of doctor ranking. The ranking is shown in Table 15.
Step 9. Propose a compromise ranking.
Table 15 shows the final results of sensitivity analysis. Let vbe the regulated value, namely,
maximal group utility (v=1), maximal regret (v=0) and combined maximal group utility and
regret (v=0.25,v=0.5,v=0.75).
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 21
Table 1 5
The ranking of doctors
v=1v=0.75 v=0.5 v=0.25 v=0
D1 1 1 1 1 1
D2 6 6 6 6 6
D3 9 8 9 9 10
D4 4 4 4 4 4
D5 8 9 10 11 11
D6 5 5 5 5 5
D7 7 7 8 8 8
D8 10 10 7 7 7
D9 12 12 12 12 12
D1033333
D1122222
D12 11 11 11 10 9
Table 1 6
Comparison between various methods
Methods Doctors ranking
Original ranking D1D2=D3=D4=D5=D6D7=D8=D9=D10 =D11 D12
The VIKOR method (v=0.5) D1D11 D10 D4D6D2D8D7D3D5D12 D9
TOPSIS D9D5D12 D11 D7D1D4D6D2D8D3D10
Sentiment analysis (v=0.5)D1D3D2D4D5D6D8D10 D11 D7D9D12
Based on the result (Table 15), it shows that the value vinfluences doctors ranking. The rank-
ing of D3,D5,D7,D8,D12 negligibly changes as the parameter vchanges; while the ranking of
D1,D10,D11 ,D4,D6,D2is always stable. Therefore, we can conclude that the VIKOR method has
strong stability on the premise of privacy protection. We can also conclude that VIKOR method
can provide stable ranking list to DMs in other cases. Thus, the proposed DRS model is practicable
and effective.
4.2. Comparative analysis and discussions
To validate our constructed model, we compare VIKOR with sentiment analysis (Boran et al., 2009)
and TOPSIS (Dong et al., 2016) by analysing the ranking results and favourable rate. Because we
use the fair term to replace sparse data, and the favourable evaluation contains no fair term. It is
not accurate for DMs to select doctors based on excellent evaluation. In view of this limitation, we
introduce favourable rate to analyse the difference between different methods. Favourable rate is
the excellent evaluation of the total evaluation, excluding missing data. The comparative results are
shown in Table 16.
Based on the result (Table 16), we get following discussions:
(1) The most optimal doctor is almost D1, except for the result of TOPSIS, which indicates the
drawback of TOPSIS method and the advantage of VIKOR method. In addition, it is easily
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
22 Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
Fig. 10. The favourable rate of doctors. [Colour figure can be viewed at wileyonlinelibrary.com]
to find that TOPSIS is unfit for our case according to the rankings of D1and D9. TOPSIS has
drawbacks in computing distance between individual and ideal evaluation, where the rankings
can be affected by DMs easily. We can also prove the validity of VIKOR method. Thus, TOPSIS
is unsuitable for risk-averse patients. While VIKOR method utilises compromising solution to
rank doctors, which overcomes the drawbacks of TOPSIS.
(2) The original ranking often appears the same ordering among alternatives (such as D2=D3=
D4=D5=D6), which makes it difficult to compare these alternatives. However, the ranking
result obtained from VIKOR method could distinguish these alternatives effectively, which
shows the validity and feasibility of proposed method.
(3) Table 16 shows that the ranking obtained by original method is similar to the ranking obtained
by sentiment analysis. And the rankings obtained from VIKOR method and sentiment analysis
offer some striking differences. Because sentiment analysis only sums up emotional offset,
whereas IFSs describe sentiment in accordance with each standard. In addition, IFSs have
some advantages, such as easy calculation and extensive application. Therefore, IFSs are more
appropriate to describe emotional offset than the other fuzzy sets.
The favourable rate of doctors is shown in Fig. 10.
By comparing Table 16 with Fig. 10, we can draw some conclusions as follows.
(1) Except for the rankings of the VIKOR method, the favourable rate and rankings of D9,D5,
D3are high; the favourable rate and rankings of D10 and D11 are nearly the worst. However,
the rankings obtained by VIKOR method are the opposite. These results show the flexibility of
VIKOR method. Therefore, we can conclude that VIKOR method proposes a proper solution
under various conditions than other methods.
(2) From above discussion, we can also conclude that the trend of change between doctors ranking
obtained by VIKOR method and favourable rate is inconsistent. While the trend of change
between doctors ranking obtained by other methods and favourable rate are similar. Clearly,
sparse data have an influence on favourable rate. So, we can conclude that the sparse data
have significant influence on doctors ranking except for the ranking obtained by VIKOR
method.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 23
In summary, the proposed model provides proper rankings according to the history evaluation,
with little impact from sparse data. Moreover, the proposed model could find certain excellent
doctors, avoiding excessive concentration of doctor resources. In practical application, our proposed
model is also suitable for solving incomplete MCGDM problems on the premise of considering
sparse data and user privacy.
5. Conclusion and future works
This study establishes a ranking model that utilises evaluation of patients including textual informa-
tion and numerical information to rank doctors. The proposed model integrates IFS with TF-IDF
and VIKOR, in order to solve MCGDM problem. We use TF-IDF to deal with textual information
and obtain criteria weights. Then IFSs are used to describe emotional offset. Finally, we propose the
VIKOR method to address MCGDM problem. In addition, by applying the doctors ranking case,
we conclude that our proposed model has strong stability and operability when solving incomplete
MCGDM problems on the premise of considering sparse data and user privacy.
This study contributes to providing more proper alternatives ranking for users than extant meth-
ods on the premise of sparse data and user privacy. From the practical application, the performance
of textual information enriches the evaluation, which can help to improve the accuracy of ranking.
The criteria weights obtained by TF-IDF help to reduce subjectivity in evaluation. The utilisation
of IFSs could describe emotional offset and uncertainty in users’ evaluation all at once. From the
results of case study, we can also conclude that the VIKOR method provides more appropriate
ranking result than the other existing approaches. Moreover, our proposed model has extensive
application in real life. From the theoretical point, integrating TF-IDF and VIKOR could address
sparse data to improve accuracy of ranking under fuzzy environments.
However, this study has some limitations that guide us to future works. First, as the textual
information increases, it is necessary to improve emotion directory and text classification technology.
Second, IFSs can only describe positive, hesitate and negative emotion. In order to improve accuracy
of ranking, we expect to introduce neutral degree. Third, the study utilises fair terms to replace sparse
data, we will try to propose a novel technology to reduce data sparsity problem in future works.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (No. 71771219)
and the open project of “Mobile Health” Ministry of Education-China Mobile Joint Laboratory of
Central South University.
References
Ardissono, L., Goy, A., Petrone, G., Segnan, M., Torasso, P., 2003. Intrigue: personalized recommendation of tourist
attractions for desktop and hand held devices. Applied Artificial Intelligence 17, 8–9, 687–714.
Asgharizadeh, E., Taghizadeh Yazdi, M., Mohammadi Balani, A., 2017. An output-oriented classification of multi-
ple attribute decision-making techniques based on fuzzy c-means clustering method. International Transactions in
Operational Research. https://doi.org/10.1111/itor.12449.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
24 Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
Atanassov, K.T., 1986. Intuitionistic fuzzy-sets. Fuzzy Sets and Systems 20, 1, 87–96.
Atanassov, K.T., 1994. New operations defined over the intuitionistic fuzzy-sets. Fuzzy Sets and Systems 61, 2, 137–142.
Boran, F.E., Genc¸, S., Kurt, M., Akay, D., 2009. A multi-criteria intuitionistic fuzzy group decision making for supplier
selection with TOPSIS method. Expert Systems with Applications 36, 8, 11363–11368.
Burillo, P., Bustince, H., 1995. Intuitionistic fuzzy relations (Part I). Mathware & Soft Computing 2, 1, 5–38.
Cao, Y.K., Li, Y.F., 2007. An intelligent fuzzy-based recommendation system for consumer electronic products. Expert
Systems with Applications 33, 1, 230–240.
Chen, G., Chen, L., 2014. Recommendation based on contextual opinions. International Conference on User Modeling,
Adaptation, and Personalization, Hong Kong, China.
Chen, G., Chen, L., 2015. Augmenting service recommender systems by incorporating contextual opinions from user
reviews. User Modeling and User-Adapted Interaction 25, 3, 295–329.
Chen, T., 2016. Ubiquitous multicriteria clinic recommendation system. Journal of Medical Systems 40, 5, 113.
Chen, Y.L., Cheng, L.C., Chuang, C.N., 2008. A group recommendation system with consideration of interactions among
group members. Expert Systems with Applications 34, 3, 2082–2090.
Dammak, F., Baccour, L., Alimi, A.M., 2015. A comparative analysis for multi-attribute decision making methods:
TOPSIS, AHP, VIKOR using intuitionistic fuzzy sets. IEEE International Conference on Fuzzy Systems, Istanbul,
Tur ke y.
De, S.K., Biswas, R., Roy, A.R., 2001. An application of intuitionistic fuzzy sets in medical diagnosis. Fuzzy Sets and
Systems 117, 2, 209–213.
Dev, H., Anjum, R., 2016. Application of text mining to build a recommendation system for restaurants and their dishes.
International Journal of Computer Applications 155, 4, 1–6.
Dong, R.H., O’mahony, M.P., Schaal, M., Mccarthy, K., Smyth, B., 2016. Combining similarity and sentiment in opinion
mining for product recommendation. Journal of Intelligent Information Systems 46, 2, 285–312.
Gao, H., Lu, M., Wei, G.W., Wei, Y., 2018a. Some novel Pythagorean fuzzy interaction aggregation operators in multiple
attribute decision making. Fundamenta Informaticae 159, 4, 385–428.
Gao, H., Wei, G.W., Huang, Y.H., 2018b. Dual hesitant bipolar fuzzy Hamacher prioritized aggregation operators in
multiple attribute decision making. IEEE Access 6, 1, 11508–11522.
Garg, H., 2016. A new generalized improved score function of interval-valued intuitionistic fuzzy sets and applications
in expert systems. Applied Soft Computing, 38, C, 988–999.
Gong, J.B., Sun, S.T., 2011. Individual doctor recommendation model on medical social network. 7th International
Conference, ADMA, Beijing, China.
Guan, Z.Y., Bu, J.J., Mei, Q.Z., Chen, C., Wang, C., 2009. Personalized tag recommendation using graph-based ranking on
multi-type interrelated objects. International ACM SIGIR Conference on Research and Development in Information
Retrieval, Boston, MA, USA.
Hao, H., 2015. The development of online doctor reviews in China: an analysis of the largest online doctor review website
in China. Journal of Medical Internet Research 17, 6, e134.
Hao, H.J., Zhang, K.P., 2016. The voice of Chinese health consumers: a text mining approach to web-based physician
reviews. Journal of Medical Internet Research 18, 5, e108.
Hao, L.L., Hao, L.Z., 2008. Automatic identification of stop words in Chinese text classification. International Conference
on Computer Science and Software Engineering, Hubei, China.
Hoens, T.R., Blanton, M., Chawla, N.V., 2010. Reliable medical recommendation systems with patient privacy. ACM
International Health Informatics Symposium, Arlington, VA, USA.
Hong, D.H., Choi, C.H., 2000. Multicriteria fuzzy decision-making problems based on vague set theory. Fuzzy Sets and
Systems 114, 1, 103–113.
Hu, J.H., Pan, L., Chen, X.H., 2017. An interval neutrosophic projection-based VIKOR method for selecting doctors.
Cognitive Computation 9, 6, 801–816.
Hu, J.H., Yang, Y., Zhang, X.L., Chen, X.H., 2018. Similarity and entropy measures for hesitant fuzzy sets. International
Transactions in Operational Research 25, 3, 857–886.
Huang, Y.M., Liu, C.H., Lee, C.Y., 2012. Designing a personalized guide recommendation system to mitigate information
overload in museum learning. Educational Technology & Society 15, 4, 150–166.
Hwang, W.-Y., 2018. Variable selection for collaborative filtering with market basket data. International Transactions in
Operational Research. https://doi.org/10.1111/itor.12518.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26 25
Jones, K.S., 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation
60, 1, 493–502.
Liu, H.W., Wang, G.J., 2007. Multi-criteria decision-making methods based on intuitionistic fuzzy sets. European Journal
of Operational Research 179, 1, 220–233.
Manikrao, U.S., Prabhakar, T.V., 2005. Dynamic selection of web services with recommendation system. International
Conference on Next Generation Web Services Practices, Seoul, Korea.
Martellos, S., 2012. From a textual checklist to an information system: the case study of ITALIC, the Information System
on Italian Lichens. Plant Biosystems 146, 4, 764–770.
Martineau, J., Finin, T., 2009. Delta TFIDF: an improved feature space for sentiment analysis. International Conference
on Weblogs and Social Media, San Jose, C, USA.
Meng, F.Y., Chen, X.H., 2015. Interval-valued intuitionistic fuzzy multi-criteria group decision making based on cross
entropy and 2-additive measures. Soft Computing 19, 7, 2071–2082.
Meng, F.Y., Chen, X.H., 2016. Entropy and similarity measure of Atanassov’s intuitionistic fuzzy sets and their application
to pattern recognition based on fuzzy measures. Pattern Analysis and Applications 19, 1, 11–20.
Paik, J.H., 2013. A novel TF-IDF weighting scheme for effective ranking. International ACM SIGIR Conference on
Research and Development in Information Retrieval, July 28–August 1, Dublin, Ireland, pp. 343–352.
Paltoglou, G., Thelwall, M., 2010. A study of information retrieval weighting schemes for sentiment analysis. ACL 2010,
Proceedings of the Meeting of the Association for Computational Linguistics, Uppsala, Sweden.
Rajarajeswari, P., Uma, N., 2014. Normalized hamming similarity measure for intuitionistic fuzzy multi sets and its
application in medical diagnosis. International Journal of Mathematics Trends and Technology 5, 3, 219–225.
Roostaee, R., Izadikhah, M., Lotfi, F.H., Rostamy-Malkhalifeh, M., 2012. A multi-criteria intuitionistic fuzzy group de-
cision making method for supplier selection with VIKOR method. International Journal of Fuzzy System Applications
2, 1, 1–17.
Schafer, J.B., Konstan, J.A., Riedl, J., 2001. E-commerce recommendation applications. Data Mining and Knowledge
Discovery 5, 1, 115–153.
Schuckert, M., Liu, X.W., Law, R., 2015. Hospitality and tourism online reviews: recent trends and future directions.
Journal of Travel & Tourism Marketing 32, 5, 608–621.
Sobrie, O., Mousseau, V., Pirlot, M., 2018. Learning monotone preferences using a majority rule sorting model. Interna-
tional Transactions in Operational Research. https://doi.org/10.1111/itor.12512.
Stern, D.T., Wojtczak, A., Schwarz, M.R., 2009. Global minimum essential requirements in medical education. Medical
Teacher 25, 6, 589–595.
Sun, R.X., Hu, J.H., Chen, X.H., 2017a. Novel single-valued neutrosophic decision-making approaches based on prospect
theory and their applications in physician selection. Soft Computing. https://doi.org/10.1007/s00500-017-2949-0.
Sun, R.X., Hu, J.H., Zhou, J.D., Chen, X.H., 2017b. A hesitant fuzzy linguistic projection-based MABAC method for
patients’ prioritization. International Journal of Fuzzy Systems. https://doi.org/10.1007/s40815-017-0345-7.
Wang, J., Wang, J.Q., Tian, Z.P., Zhao, D.Y., 2018a. In a multi-hesitant fuzzy linguistic multi-criteria decision-making
approach for logistics outsourcing with incomplete weight information. International Transactions in Operational
Research 25, 3, 831–856.
Wang, J.Q., Zhang, H.Y., 2013. Multicriteria decision-making approach based on Atanassov’s intuitionistic fuzzy sets
with incomplete certain information on weights. IEEE Transactions on Fuzzy Systems 21, 3, 510–515.
Wang, J.Q., Zhang, X., Zhang, H.Y., 2018b. Hotel recommendation approach based on the online consumer reviews
using interval neutrosophic linguistic numbers. Journal of Intelligent & Fuzzy Systems 34, 1, 381–394.
Wei, G.W., 2010. GRA method for multiple attribute decision making with incomplete weight information in intuitionistic
fuzzy setting. Knowledge-Based Systems 23, 3, 243–247.
Wei, G.W., 2018a. Picture fuzzy Hamacher aggregation operators and their application to multiple attribute decision
making. Fundamenta Informaticae 157, 3, 271–320.
Wei, G.W., 2018b. Some similarity measures for picture fuzzy sets and their applications. Iranian Journal of Fuzzy Systems
15, 1, 77–89.
Wei, G.W., Gao, H., 2018. The generalized dice similarity measures for picture fuzzy sets and their applications. Informatica
29, 1, 1–18.
Wei, G.W., Lu, M., 2018. Pythagorean fuzzy power aggregation operators in multiple attribute decision making. Interna-
tional Journal of Intelligent Systems 33, 1, 169–186.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
26 Hu et al. / Intl. Trans. in Op. Res. 00 (2018) 1–26
Wei, G.W., Wei, Y., 2018. Similarity measures of Pythagorean fuzzy sets based on the cosine function and their applications.
International Journal of Intelligent Systems 33, 3, 634–652.
Weng, S.S., Liu, M.J., 2004. Personalized product recommendation in e-commerce. IEEE International Conference on
E-Technology, E-Commerce and E-Service, Taipei, Taiwan.
W´
ojtowicz, A., ˙
Zywica, P., Stachowiak, A., Dyczkowski, K., 2016. Solving the problem of incomplete data in medical
diagnosis via interval modeling. Applied Soft Computing 47, 424–437.
Wu, H.C., Luk, R.W.P., Wong, K.F., Kwok, K.L., 2008. Interpreting TF-IDF term weights as making relevance decisions.
ACM Transactions on Information Systems 26, 3, 55–59.
Xu, Z.S., 2007. Methods for aggregating interval-valued intuitionistic fuzzy information and their application to decision
making. Kongzhi Yu Juece/Control & Decision 22, 2, 215–219.
Yager, R.R., 2004. OWA aggregation over a continuous interval argument with applications to decision making. IEEE
Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 34, 5, 1952–1963.
Yu, S.M., Wang, J., Wang, J.Q., 2018. An extended TODIM approach with intuitionistic linguistic numbers. International
Transactions in Operational Research 25, 3, 781–805.
Yu, S.M., Wang, J., Wang, J.Q., Lin, L., 2017. A multi-criteria decision-making model for hotel selection with linguistic
distribution assessments. Applied Soft Computing 67, 741–755.
Zadeh, L.A., 1965. Fuzzy sets. Information & Control 8, 3, 338–353.
Zhang, H.Y., Ji, P., Wang, J.Q., Chen, X.H., 2017. A novel decision support model for satisfactory restaurants utilizing
social information: a case study of TripAdvisor.com. Tourism Management 59, 281–297.
Zhang, Y., 2016. iDoctor: personalized and professionalized medical recommendations based on hybrid matrix factor-
ization. Future Generation Computer Systems, 66, 30–35.
C
2018 The Authors.
International Transactions in Operational Research C
2018 International Federation of Operational Research Societies
