An Opinion Mining Task in Turkish Language: A Model for Assigning Opinions in Turkish Blogs to the Polarities *

Abstract

In our day, social networks are also called as the new generation media, which give the daily increasing users the change to gather at these environments and thus interactive environments get created with the messages noteworthy enough to share. Thereby, the cooperation of big masses and joint production are rendered possible and new tendencies are provided a change. Thanks to these environments, users improved their position from "content consumer" to "content creator". To wit, while the users share their views on the product/service with the ones they know and their close ones use the spreading by word of mouth method, they've already started to cover these environments as well. These environments, which in a way globalize the customer opinions, are analysis tools for businesses due to customer focused structuring of our day. By analyzing the data they acquire from such environments, businesses are required to develop suitable strategies Marmara University, Istanbul, Turkey Global changes took place at a neck-breaking speed in lots of fields along with the Web 2.0 era, which can be stated as the new Internet trend. Web pages which once were a statical structure that can be said to become dynamic pages created by users, and in this regard they can be said to have been democratized by evolving. Social media, which were structured alongside with this era, by providing a large data flow for businesses, present new and improvable opportunities in the field of creating effective strategies. There are lots of blogs in today's Internet environment which includes customer ideas regarding the products/services that they possess. This environment, which in a way globalizes the customer ideas, is a new medium suitable for examination in terms of its increasing the business-customer interaction and due to its transporter nature; it provides the text data that may be analyzed in the field of Customer Relationship Management to businesses. Thus, businesses should follow blog environments to see how the product/service they provide is greeted in terms of the customer focus and it should be seen as an important job on which they can conduct effective analyses. For this purpose, a model proposal that will assign the ideas to the Turkish blogs was given in the study. Opinion mining methods were used in the model, and so to perceive a general look-on about products/services, a methodology was devised, which will assign the text based opinion data on the Turkish blogs to the poles. Success of the pole assignment of the model is evaluated with the precision measure.

Full text

Journalism and Mass Communication, ISSN 2160-6579
March 2013, Vol. 3, No. 3, 179-198
An Opinion Mining Task in Turkish Language: A Model for
Assigning Opinions in Turkish Blogs to the Polarities∗
Çiğdem Aytekin
In our day, social networks are also called as the new generation media, which give the daily increasing
users the change to gather at these environments and thus interactive environments get created with the
messages noteworthy enough to share. Thereby, the cooperation of big masses and joint production are
rendered possible and new tendencies are provided a change. Thanks to these environments, users improved
their position from “content consumer” to “content creator”. To wit, while the users share their views on the
product/service with the ones they know and their close ones use the spreading by word of mouth method,
they’ve already started to cover these environments as well. These environments, which in a way globalize the
customer opinions, are analysis tools for businesses due to customer focused structuring of our day. By
analyzing the data they acquire from such environments, businesses are required to develop suitable strategies
Marmara University, Istanbul, Turkey
Global changes took place at a neck-breaking speed in lots of fields along with the Web 2.0 era, which can be stated
as the new Internet trend. Web pages which once were a statical structure that can be said to become dynamic pages
created by users, and in this regard they can be said to have been democratized by evolving. Social media, which
were structured alongside with this era, by providing a large data flow for businesses, present new and improvable
opportunities in the field of creating effective strategies. There are lots of blogs in today’s Internet environment
which includes customer ideas regarding the products/services that they possess. This environment, which in a way
globalizes the customer ideas, is a new medium suitable for examination in terms of its increasing the
business-customer interaction and due to its transporter nature; it provides the text data that may be analyzed in the
field of Customer Relationship Management to businesses. Thus, businesses should follow blog environments to
see how the product/service they provide is greeted in terms of the customer focus and it should be seen as an
important job on which they can conduct effective analyses. For this purpose, a model proposal that will assign the
ideas to the Turkish blogs was given in the study. Opinion mining methods were used in the model, and so to
perceive a general look-on about products/services, a methodology was devised, which will assign the text based
opinion data on the Turkish blogs to the poles. Success of the pole assignment of the model is evaluated with the
precision measure.
Keywords: opinion mining, text classification, sentiment classification, semantic orientation, positive/negative
polarity
Introduction
∗ Acknowledgments: The author would like to thank Sepandar Kamvar for allowing me to use the color codes regarding
sentiment words in We Feel Fine web site. The author really appreciates that.
Çiğdem Aytekin, Ph.D., Faculty of Engineering, Marmara University.
D AV I D PUBLISHING
D

AN OPINION MINING TASK IN TURKISH LANGUAGE
180
as accordance with the customer choices and to update these strategies constantly.
On the other hand, these environments that include the customer opinions on the product/service possess
text data and these data aren’t structured the same way as the data saved into the database are structured.
Significant information is procured by analyzing the structured data with the data mining techniques in our day.
Yet, this form of the text data, that said the unstructured form, hardens the knowledge extraction from different
topics. But beneficial the text data increase constantly and in lots of cases, the knowledge that can be extracted
from these data is needed. Examination of these data manually is very hard, moreover, it is impossible in most
cases. The need of analyzing of the text data and the desire to lessen the amount of time wasted manually
created a structuring in this direction and thus, the concept text mining came to be. The text data become
structured with the text mining method and thanks to refinement after being analyzed with data mining
techniques, procurement process of the precious knowledge can be realized. Therefore, secret and unknown
meanings among the text data can be derived. And when the mentioned text data are opinion-stating statements,
tasks of opinion mining, which is an implementation field of the text mining, come into prominence.
Generally, opinion mining is the transition of automatically knowledge obtaining from the text data that
state opinion. In this context, as stated in the Wikipedia (Sentiment Analysis, 2013) sources, opinion mining
refers to a large field of Computational Linguistics, Text Mining, and Natural Language Processing. And when
the opinion mining explanations in the literature are examined, it can be seen that different authors state
opinions in the same parallel.
CHEN, ZHU, Kifer, and Lee (2010) based the opinion mining on the essence of the question “What do
people think about …?”. According to them, this question becomes more important in the web platforms where
people state their opinions. As people share their positive and negative opinions regarding products in these
environments, they actually end up being a representative on the topic of general opinions regarding the
product. These data belonging to the consumer experience can be analyzed with opinion mining methods, and
consumption behaviors of new customers can be affected from these opinions.
Conrad and Schilder (2007) evaluated the customer opinions on the product/service within the scope of
both the individual and businesses. According to them, individually-based consumer candidates may want to
gain a general perspective regarding a product they want to purchase or the service itself. This need also applies
to businesses. Similarly, they may also wish to gain a general view concerning how the service they provide or
the product they present are greeted.
LIU (2013) also approached the topic from two angles just as Conrad and Schilder did (2007). For him,
consumer experiences are vital in taking deliberative decisions on individual basis. With Opinion Mining
implementations, a general perspective about the product or service can be obtained without reading lots of
comments. Similarly, opinion mining is important for business as well. For example, a business must know
how its product or service is perceived by the consumers. Not just that, it also should know the perspective
regarding the rival products. Thus, it can conduct the necessary evaluations concerning the product or service
and innovations on different topics can be developed on this direction.
While Falcon (2010) stated the opinion mining as “a technology whose validity was proved”, he
emphasized on it using statistical model and software and stated that it had been used widely as a listening,
analysis, and relation tool in the defence, public administration, and marketing practices.
Works to be done with opinion mining can be evaluated within the tasks of opinion mining. As such, these
tasks have been focused on below.

AN OPINION MINING TASK IN TURKISH LANGUAGE
181
Opinion Mining Tasks
When the literature studies about the opinion mining tasks are examined, though different authors seem to
conduct evaluations within the frame of different categories, there are some common points. The reason why
these differences occur mostly bases on the purpose. Some certain opinion mining tasks were given below:
(1) Determining the positive-negative (PN) pole of the text. This is one of the most commonly-used tasks
of opinion mining. In this task, which of the positive or negative poles do the user opinions partake in is tried to
be determined. For example, answer to the question “Is the general take of the users on our products/services
positive or negative?” could be tried to find. Esuli and Sebastiani (2006), LIU (2007a), and Abbasi, CHEN, and
Salem (2008) are example authors who take up this task.
(2) Determining the intensity of the PN pole text. In this task, the text based opinion data that were
assigned to the PN poles are ranked on certain levels. For example, answer to the question “How many of the
opinions which approach positively/negatively to our products/services are poorly negative/positive, how many
are moderately negative/positive and how many are strongly positive/negative?” could be tried to find. Esuli
and Sebastiani (2006) is an example author who takes up this task.
(3) Feature-based mining. In this task, whether the findings regarding the features of the products/services
which users possess are positive or negative is tried to be determined. For example, answer to the question
“What do the users think about the Y feature of our X product/service?” could be tried to find. Leneve (2010),
Verbeke and Eynde (2013), and QIU, LIU, BU, and CHEN (2010) are example authors who take up this task.
(4) Comparative sentence and relation deduction. In this task, comparative relation is tried to be deducted.
By comparing an object with another, differences are determined. For example, evaluations regarding a brand
being superior to another one can be deducted from the sentence “X brand car is cheaper than Y brand car”.
Leneve (2010), LIU (2007b), and Verbeke and Eynde (2013) are example authors who take up this task.
(5) Mentioning an issue as classification problem. In this task, an issue gets mentioned and conducting a
classification regarding this issue is tried. For example, a classification can be done concerning the person who
deserts the brand in the sentence that “I will never an x brand product”. Verbeke and Eynde (2013) are example
authors who take up this task.
On the other side, the opinion mining task regarding the subjectivity-objectivity of the text is one of the
tasks partaking in the literature. Yet, it didn’t receive as much attention as others. Due to the nature of opinion
mining, text data being statements of opinion and these possessing a rightful subjectivity caused this to happen.
For the study is about determining the positive/negative pole of a Turkish opinion, this classification task
is examined below in a more detailed way within the frame of literature studies.
Literature Review
Classification of the text data that state opinion has been the task on which, within the opinion mining
tasks, most number of applications was developed. And most of these classifications base upon
positive/negative pole differentiation. Yet, there are lots of hardships faced in the opinion mining. An important
portion of these is within the scope of spell check as stated in the fourth part of the text—“A model for
Assigning Opinions in Turkish Blogs to the Polarities”, and yet while the users write comments in which they
state their opinions in web platforms, they generally use colloquial language. Because these environments are
based upon an informal structure due to their nature. Stavrianou and Chauchat (2008) also emphasized on this

AN OPINION MINING TASK IN TURKISH LANGUAGE
182
problem and defined it as the difference between “usage regarding the colloquial language” and “usage in an
online newspaper”. Similarly, Pang and Lee (2008) stated that text behavior regarding the opinion would be
different that classic text mining applications.
When the literature is checked in respect to classifying the opinion, it is seen that supervised, unsupervised,
and semi-supervised learning techniques are the topics of subjects. For example, Turney (2002) used
unsupervised learning techniques. In this study, he used some adjective and adverb based words as his focus
and classified the user comments as “suggested ones” and “non-suggested ones”. Method has an accuracy value
of 74% on average.
Another example is the study of Devitt and Ahmad (2007) in the field of financial news. But, here, in
addition to determining the pole of a news text, establishing the pole intensity was also tried. As mentioned in
the second part of the text—“Opinion Mining Tasks”, this is a different opinion mining task. In this direction, a
scale was devised in the study to check how positive or negative the text is. Scale consists from seven levels
from too positive to too negative. Suggested method has an accuracy value of 46% in determining the positive
and negative differentiation.
Dave, Lawrence, and Pennock (2003) combined the scores from the education set to classify the
positive/negative product comments with the method of choosing of the proper features and metrics and to this
end, they put forth various techniques. The highest accuracy value of the method, in which text mining
algorithms, such as Naive Bayes and Support Vector Machine are given comparatively, is 85.3%.
Esuli and Sebastiani (2005) suggested a method based on a semi-supervised learning basis so to determine
the positive/negative orientation of the terms. According to them, definitions of similarly oriented terms tend to
be similar. Method which makes use of the lexical relations such as synonymy, antonymy, and hypernymy is
based on WordNet. Accuracy value of the method is given separately for three classifiers: 83%, 87%, and 88%.
Lastly, Kamps, Marx, Mokken, and De Rijke (2004), in their studies where they used WordNet to measure
the semantic orientation of adjectives, made the suggestion of interpreting the topics on scales such as
positive/negative, ugly/beautiful, powerful/weak, excellent/poor, and active/passive. Accuracy value of the
method is 67.18%.
In this study, opinion classification task of opinion mining was given place and the suggestion of “a model
which will assign the customer post/comments on Turkish Blogs to the poles positively or negatively” was
presented. Here, the accuracy value is 72.71% and this will be discussed in the fourth part of the text—“A
model for Assigning Opinions in Turkish Blogs to the Polarities”.
A Model for Assigning Opinions in Turkish Blogs to the Polarities
The new media used in this study is blogs. Blogs include user opinions that state sincere comments and
sentiments regarding a specific subject. Besides, since the blogger and the user are voluntarily in this medium,
clear and transparent user views, which are difficult to obtain through other ways, are reached easier. Blogs,
which gather individuals who have similar interests, include millions of comments and their number is
increasing day after day. Problems, expectations, approaches, complaints, all of them, can be found in this
medium. Blogs are considered to be a “media that is developed by the consumers”. For this very reason, the
text mass in the blogs include crucial data. Therefore, enterprises should follow the blogs in order to learn how
the products and services they offer are received by the customers, and they should see the blogs as an
important medium through which they can perform an effective analysis.

AN OPINION MINING TASK IN TURKISH LANGUAGE
183
On the other hand, the comments in this medium are crucial sources of feedback since they are stated by
the users themselves. However, feedback is limited in the traditional mass media. The information gathered
from this medium will offer a solution for the limitation in feedback, and enterprises will be able to increase
efficiency by developing the strategies regarding enterprise decision process. In addition, while the data on the
enterprise’s database are limited to the use of the enterprise, these blogs are open to anyone. Therefore, positive
or negative opinions can be seen by anyone who reads that blog. For this reason, enterprises have to follow and
analyze blogs in order to protect their reputations.
In this regard, this study proposes a model that will assign the text-based opinion data in Turkish blogs to
positive and negative polarities in order to present a general view on products and services. The model is a
semi-supervised learning model. Training set comprises of English words stating sentiments. In order to
calculate the words’ probability for being in positive or negative polarities, color codes that have been assigned
to the words were used and through a repetitive test-investigation process, color-word meaning correlation was
provided.
In order to perform polarity assignment process automatically, a program called “Opinion Polarity
Detection” was developed. For this program, Visual Basic Language was used in Microsoft Net Framework
environment and Microsoft SQL Server 2005 was used as database. The program classifies the text data
according to the rules of Naive Bayes Algorithm, which is simple to use and produces efficient results. For
spell-check, the error correction module of the Microsoft Word text editor was used. Besides, posts’ and
comments’ length in terms of word number were analyzed and its effect on the success of polarity assignment
was studied in the study.
Preparation of Data
The data in the model comprise of the customer opinions gathered from Turkish blogs. These text data can
be blogger’s posts regarding the comments of the readers as well they can be the readers’ comments on the
relevant post.
A test database was developed especially for the aim of the study. This database is a sample group
comprising of 350 positive and 350 negative posts/comments. In the test database, posts/comments regarding
the products and services under the headings of “white goods, built-in products, electronics, small home
appliances, and heaters-air conditioners” were included. The most important factor for the selection of this
heading is that the products give a chance for the posts and comments regarding services as well as the products
themselves. On the other hand, using this heading will enable the posts and comments regarding both domestic
and foreign products and services. While searching for posts/comments, Google Blog Search Engine was used
in Turkish language and using brand names as keyword was found to be easiest way for finding text data
regarding the enterprise in question. While searching for the post/comment; instead of searching for the posts
regarding especially one single brand, the search was tried to be diversified through searching for as many
brands as possible. In addition to these, following points were of importance in the posts/comments collected:
(1) Both long and short entries were included in the posts/comments. While the average word number of
the negative post/comments was 86.66, and the average for the positive entries was 29.66;
(2) In data collecting, to the introductions that are evaluated semantically in positive or negative pole, a
part was given;
(3) The posts/comments which included positive opinions for a property of the product and service and

AN OPINION MINING TASK IN TURKISH LANGUAGE
184
negative opinions for other properties were not included. Yet, post/comment should take place in only one of
the positive or negative poles semantically in terms of its every feature;
(4) Some of the collected posts/comments might be placed in the positive/negative polarity for only a
single property while some might be placed in the related polarity for more than one property. In this regard,
the posts/comments were not limited;
(5) The data in the posts/comments might be texts that are related to the properties of the products and
services as well as they might be texts that are related to the enterprise’s approach. The data were not
differentiated in this sense;
(6) The negative posts/comments were text data including complaints in terms of discontent and
dissatisfaction while the positive posts/comments were text data including recommendations in terms of content
and satisfaction;
(7) The posts/comments were not differentiated based on time such as day, month, or year.
In order to ease collecting the posts/comments and transferring them to the program, a form in the
Personal Home Page (PHP) language was developed and it was ensured that this form was uploaded on the
Web after a domain name, “data collection form”, was taken. The posts/comments found through the search
engine on the basis of the points mentioned above were entered into this from. The text data which were
manually determined to be in the positive/negative polarity were copied into this from with their Uniform
Resource Locator (URL) addresses and a database was developed.
Setting up of the Model
XU and CHENG (2011), who have greatly contributed to the literature studies on opinion mining,
correlated the words “opinion” and “sentiment” in this way. “An opinion contains often sentiment words which
can be classified into polarities such as positive, negative, and neutral” (p. 11). From this point, it was assumed
that the sentiment words could be used as the training set in polarity determination and as the result of the
performed researches it was decided that the sentiment indicator words in the website called We Feel Fine
would be used. In this way, a semi-supervised learning model was developed.
(1) English training set for semi-supervised learning.
Training set comprises of 2,178 sentiment indicator words that can be found in
http://www.wefeelfine.org/data/files/feelings.txt. The methodology of the website can be summarized in this
way (the explanations are limited to the information given in http://www.wefeelfine.org/methodology.html).
Data collection: We Feel Fine website includes a “collecting motor” which collects human feelings
automatically from online sources such as LiveJournal, MSN Spaces, MySpace, Blogger, Flickr, and
Technorati. The purpose is to find the statements “I feel” or “I am feeling” in the blog posts. These statements
which are saved into the database get checked to see if they include valid feelings. Valid feelings consist from
adjective/adverb based words, which are structured manually and determined beforehand, and there are 5000 of
them. If a statement in the database includes even if a single one from these 5000 words, then it gets saved into
the real database along with the name of the blogger. By sorting out the blogger’s name from the URL address,
that blogger’s profile info can be accessed via blogging companies. Thus info about the blogger regarding his
age, gender, city, country, and even the weather conditions at the time of his post becomes saved into this
database. Repetition period of this process is 10 minutes.
Statistical measurements: We Fell Fine database can be questioned by the ones who will use its balls.

AN OPINION MINING TASK IN TURKISH LANGUAGE
185
When the application begins, balls scatter around the screen and when one of these balls are clicked, statements
including “I feel” or “I am feeling” appear. This screen is given in Figure 1. Also, screening conditions
according to the gender, age, weather condition, location, date, and feelings can be created.
Colors of feeling: Color equivalents of feelings are based on this: Bright yellow colors represent happy
positive feeling; dark blue colors represent sorrowful negative feeling, green color the feeling of calmness, and
red represents angry feelings.
Figure 1. We Feel Fine balls in the opening screen. Resource: Retrieved February 28, 2013, from
http://www.wefeelfine.org/wefeelfine_pc.html.
(2) Using Sentiment Dictionary in Turkish generated by English training set.
The full list of the 2,178 valid sentiments are given with their frequencies and assigned hexa color codes in
the URL address of the We Feel Fine website below. A part of this list is presented in the Table 1.
It was stated above that the words which formed the sentiment list were basically adverbs and adjectives.
Of all the 2,178 words in this list, 1,655 words that were used as adjective and/or adverb in Turkish were taken,
and in total 4,744 Turkish word/word groups were obtained because a word can have more than one counterpart
as adjective and/or adverb in Turkish. This dictionary comprising of 4,744 lines will be referred as “Sentiment
Dictionary in Turkish” from now on. However, it should be noted that the meanings of some English words
cannot be given with one Turkish word. For example, the word “best” is translated as “en iyi”, “en uygun”, …
with the superlative adjective “en”. For this reason, the Sentiment Dictionary in Turkish comprises of translated
words or word groups.

AN OPINION MINING TASK IN TURKISH LANGUAGE
186
Table 1
Some of the Sentiment Words in English.
Word Frequency Assigned hexa color code
better 128,155 FFA401
bad 93,390 07548A
good 76,610 FFF700
right 40,683 E97802
guilty 31,591 004E6F
sick 27,706 2E9127
same 25,389 017E94
sorry 23,779 00696F
well 22,428 E6C637
down 20,847 18213E
alone 17,988 595884
happy 17,849 FF7F00
Note. Retrieved February 28, 2013, from http://www.wefeelfine.org/data/files/feelings.txt.
Adjectives (“Adjectives”, 2007) come before nouns and qualify or determine them in different ways.
Adjectives are divided into those groups in terms of task and meaning:
(1) Qualificative adjectives;
(2) Determinative adjectives;
(3) Demonstrative adjectives;
(4) Numeral adjectives: cardinal numeral adjectives, ordinal numeral adjectives, fractional numeral
adjectives, distributive numeral adjective, and group number adjective;
(5) Indefinite adjectives;
(6) Interrogative adjectives.
The adjectives in the Sentiment Dictionary in Turkish can be classified in terms of their types as follows:
Most of them were qualificative adjectives (“sakin”, “berrak”, “iyi”, “sözlü”, “rizikolu”, … ), a small part of
them were indefinite adjectives (“bütün”, “başka”, … ), two of them were ordinal numeral adjectives
(“birinci”, “ikinci”) and one of them was a group number adjective (“ikiz”). Due to the nature of text mining,
cardinal numeral adjectives (“bir”, “iki”, … ) were not included and they were cleaned from the English
dictionary having 2,178 words.
In terms of structure, adjectives (“Adjectives”, 2007) are divided into five groups: absolute adjectives,
derived adjectives, compound adjectives, enhanced adjectives, and adjectives as word/word groups. “Kırmızı”,
“iri”, “ufak”, … can be given as examples of the absolute adjectives in the Sentiment Dictionary in Turkish;
“ağlayan”, “oynayan”, “evsiz”, “çarpıklık”, and “yuvarlak” can be given as examples of derived adjectives; and
“cana yakın”, “yurtsever”, “vatanperver”, “kısa boylu”, “ustalık”, and “evsiz barksız” can be given as examples
of compound adjectives.
At this point, it is useful to explain derived adjective type further. If an addition is made to the verb stem,
this type of derived adjective is called verbal adjective. Kurt (1998) defines verbal adjective in this way: The
words that resemble verbs since they indicate action and that are considered to be adjectives since they form an
adjective clause by qualifying a noun are called verbal adjectives. Verbal adjectives are derived with the affixes
“-en, -esi, -mez, -(a)r, -dik, -ecek, and -miş” which are added to verb stems: giden çocuk, yıkılası şehir, pişmez

AN OPINION MINING TASK IN TURKISH LANGUAGE
187
et, bakar kör, çalmadık kapı, gelecek yıl, sıkılmış limon, etc.. Some of the words given in the English training
set have taken the affixes “ing” and “ed”. These word/word groups that correspond to the verbal adjective in
Turkish were translated by adding the affixes “-en, -miş” to the ends of the verbs with which they were used.
The examples of the English words that were translated in this way are given in the Table 2.
Table 2
Examples of Verbal Adjectives in English Training Set
English word Turkish verbal adjective
crying ağlayan
playing oynayan
wasting israf edilen
growing artan
changing değişen
turning dönen
asking soran
pushing iten
acting hareket eden
loved sevilen
accomplished başarılmış
used alışılmış
overwhelmed bunalmış
stressed baskı yapılmış
trapped set çekilmiş
However, in the training set, there are some English words which cannot be translated as verbal adjectives
using the affixes “-en, -miş” although they have taken the affixes “ing/ed”: beginning-başlangıç, tired-yorgun, etc.
Adverbs, on the other hand, are words that affect the meanings of verbs, gerundials, adjectives, or adverbs
in various ways (place, time, manner, amount, and interrogative), that determine and grade them. Most of the
adverbs can be used as adjectives or nouns. However, while adjective comes before the noun and qualifies or
determines it, adverb does not come before the noun (“Adverbs”, 2007). The examples of adverbs in the
Sentiment Dictionary in Turkish are presented in Table 3 according to adverb kinds.
Table 3
Adverb Examples From the Sentiment Dictionary in Turkish According to Adverb Kinds
Adverbs of manner Adverbs of time Adverbs of place Adverbs of amount
eğri öğleden sonra ileri pek
çocukça evvela ileri doğru çok
az yükle sabah geri fazla
tıklım tıklım geç geride azıcık
abartarak sürekli aşağı çok az
övünerek ilkin aşağıya sık
gül hemen yukarıya Seyrek
hiç üst kata
elbette
ne olursa olsun
belki

AN OPINION MINING TASK IN TURKISH LANGUAGE
188
(3) Determining the polarity probability of the words using the assigned color codes.
The fact that the model established in order to determine polarities was a semi-supervised learning model
was stated in the beginning of this section. In supervised learning, the texts that determine the class are used in
order to train the system. In other words, the system is trained for finding the class of the new texts
automatically, based on the given text. The vector of the text whose class is to be found is mixed with text
mining algorithms and is assigned to the class to which it is related. However, in this study there are no polarity
indicator texts to train the system. Nevertheless, there are sentiment word/word groups associated with the
colors that will be used to determine the polarity. The model was considered to be a “semi-supervised learning
model” for this reason. The probability of each word/word group in the Sentiment Dictionary in Turkish for
being in positive/negative polarity was determined considering the color to which it was assigned and these
probabilities were used in Naive Bayes Algorithm.
In regard to the determination of probability, firstly, word/word groups’ fill colors regarding the hexa
color codes given in the English Training Set were formed. In order to do this, RGB color codes corresponding
to the hexa code were used. Hexa codes were transformed into RGB color codes using transforming programs
in order to be able to form fill colors.
RGB codes is a color space named after the first letters of the words “red-green-blue”
(“kırmızı-yeşil-mavi”) in English and is used frequently. Based on the light, the codes of all colors in the nature
are specified with reference to these three basic colors. When each color is mixed 100%, white is obtained,
when mixed 0%, black is obtained (RGB Color Space, 2013), take FFFF4F hexa code as an example. Here, FF
represents red, FF represents green, and 4F represents blue. When FFFF4F hexa code is transformed into RGB
code, the value 255 is obtained for red, the value 255 is obtained for green, and the value 79 is obtained for blue.
In RGB code, each of these three colors has a value between 0-255.
The fill colors regarding the all word/word groups in the Sentiment Dictionary in Turkish were obtained in
Microsoft Excel Program using RGB codes. The sentiment words updated with fill colors will be referred as the
“Sentiment Database in Turkish” from now on. In the database, there are 108 different hexa codes.
In the next step, HSL (hue-saturation-lightness) color codes of the word/word groups in the Sentiment
Database in Turkish were obtained. While obtaining HSL codes using hexa or RGB codes, transforming
programs were used again.
HSL is a coding system which has values separately for every single hue-saturation-lightness feature of the
color. Hue is a feature which can take up a value between 0o-360o and determines the place of the color on the
wheel. It starts with red at 0o and ends with red at 360o. Saturation stated as “%” shows the dullness and brightness
of the color. One hundred percent is the maximum value. It comes close to color grey with low color density and
at 0% grey color scale takes effect. And in the lightness feature which defines the lightness and darkness of the
color, while high L value means more white, low L value means more black, which means defining darkness
(How to Calculate a Complementary Colour (Inc. RGB/HSL Conversion)). In Table 4, a part of the Sentiment
Database in Turkish in which fill colors and HSL codes are included is presented. While calculating the
probability of each of these word/word groups for being in positive/negative polarity, these HSL codes will be
used.
The hypothesis regarding the calculation of the distance between colors: The color belonging to the
word/word group which has the most positive value in terms of meaning is determined as the “starting point
color”. The distance between the colors belonging to other word/word groups and the starting point color is

AN OPINION MINING TASK IN TURKISH LANGUAGE
189
calculated. Therefore, while the word/word group belonging to the starting point color is in positive polarity
with 100% probability, other colors and the word/word groups corresponding to these colors will gradually
become distant from this point and their probability for being at positive polarity will decrease.
Table 4
A Part of the Sentiment Database in Turkish With Fill Colors and HSL Codes
Sentiment words Frequencies Hexa code Fill color H S L
en iyi 6,433 FFFF4F 60 100 65
hoş 76,610 FFF700 58 100 50
katkısız 3,098 FFD93C 48 100 62
büyük 17,058 FFD801 51 100 50
gururlu 3,604 FFB300 42 100 50
dayanıklı 4,128 FFB200 42 100 50
daha çok 128,155 FFA401 39 100 50
geçer 4,615 FF7F1C 26 100 55
bahtiyar 17,849 FF7F00 30 100 50
etkili 5,296 FF6600 24 100 50
cana yakın 3,884 FF4B02 17 100 50
acılı 3,283 FF1A00 6 100 50
kısmetli 5,716 FED73C 48 99 62
cümbüş 1,223 FEA740 33 99 62
başarılmış 5,312 FEA53F 32 99 62
bütün 8,151 FE992D 31 99 59
belirli 3,826 FE9901 36 99 50
The word/word group that may have the most positive value in terms of meaning in the Sentiment
Database in Turkish was determined to be the Turkish correspondence of “best” and the 255-255-79 fill color
determined by the RGB code was selected as the starting point color. This color was “bright yellow” and it
coincided with the fact that the positive sentiments in We Feel Fine website were bright yellow. The HSL code
of the starting point color was 60-100-65. The value of H (60) indicated that the color was yellow, the value of
S (100) indicated that it was the brightest color, which means that it was in the most outer circle of the wheel
and the value of L (65) indicated that the 50% pure color value converged to white. In Figure 2, the place of the
starting point color in the color wheel is presented. In the HSL code, H indicates 60° and S indicates 100%. The
L value will be used later.
Once the place of the starting point color in the color wheel is determined, the distance between the other
colors, whose HSL values are known, and this point can be calculated. The calculation has two steps:
Calculating the length of the third edge of the triangle using H and S values;
Calculating the length of the hypotenuse of the right angled triangle using the length of the third edge and
L value.
For example, calculating the distance between the color belonging to the word “ayrı (different)” and the
starting point color.
Calculating the length of the third edge of the triangle using H and S values: For this word, H = 358, S = 81. In
order to calculate the distance between two points, a triangle is formed through connecting the points with the
central point of the wheel and with each other. Two edge lengths of triangle are known and equal to the S
values of the HSL codes. In addition, the angle can be calculated using the H values of the HSL codes. Once

AN OPINION MINING TASK IN TURKISH LANGUAGE
190
two edge lengths of the triangle and the angle between these two edges are known, the third edge length can be
calculated using the Cosine Law.
Figure 2. Place of the starting point color in the color wheel.
Calculating the length of the hypotenuse of the right angled triangle using the length of the third edge and
L value: In the first step, calculation was made using the H and S values of the two points in HSL color code
but L values were not included in the calculation. However, L values carry colors to a different point in vertical
plane.
The middle point of the height of the cylinder corresponds to the L value of 50% and represents the pure
color. While the ratio of 100% transforms the L value to white, the ratio of 0% transforms it to black. On the
other hand, when H and S values are 0, only L values matter and at this point, grey color scale from white to
black starts. In grey color scale, R, G, and B values are equal. For example, the HSL code of the color grey
corresponding to the values R = 150, G = 150 and B = 150 is 0°, 0%, and 69%. Grey color scale can be seen in
Figure 3 with its values of lightness and dullness.
Figure 3. Grey color scale.
Dullness (0%)
Lightness (100%) (100100100)
Pure color (50%)

AN OPINION MINING TASK IN TURKISH LANGUAGE
191
For example, the L value of the word “ayrı” is 55 while the L value of the starting point color is 65. The L
value of the starting point color (65) is the maximum value in the Sentiment Database in Turkish. Therefore,
the wheel belonging to the L = 65 value will always be on top of the other word/word groups. In this case, a
perpendicular can be lowered to the all wheels that have different L values from the starting point color. On the
other hand, the third edge length calculated using H and S values in the first step will form the other edge of the
right-angled triangle. Therefore, the length of the hypotenuse can be calculated and the distance between the
word “ayrı” and the starting point color can be found.
Now for 4,744 word/word groups, the probability of each for being in the positive polarity can be
calculated using the above calculations. In the database, the probability for being in the positive polarity was
taken as basis since when the probability for being in the positive polarity is subtracted from 100, the
probability for being in the negative polarity can be obtained. Firstly, the Sentiment Database in Turkish was
put in order from greater to lesser in terms of the distance to the starting point color. At this point, it was
assumed that the word/word group regarding the starting point color would be in the positive polarity with the
probability of 100% and the word/word group in the last line of the database would be in the positive polarity
with the probability of 1% (since the result of the 0% probability value in the algorithm calculation would be 0
due to the absorbing element, the probability was taken to be 1%). In order to find the other probabilities, a
coefficient had to be obtained according to these two values.
When looked to the color scale in the database put in order according to distance, it was found that it
coincided greatly with the explanation stated in the beginning of this section: “Happy positive feelings are
bright yellow. Sad negative feelings are dark blue. Angry feelings are red. Calm feelings are green”. In other
words, the opposite polarities were put in order between the colors “bright yellow” and “dark blue”. Therefore,
the probabilities might be put in order between 100% and 1%.
The most appropriate values regarding the probabilities of 4,744 word/word groups for being in the
positive class for the interval between 100%-1% were obtained with the coefficient 0.5159. When the distance
to the starting point color was multiplied with this coefficient and subtracted from 100, the values 99.9% and
1.01% were found, and these values were the values that were closest to the interval between 100%-1.106%
unique probability values were calculated in this way. The probability values obtained were divided into 100
and values lesser than 1 were obtained and these values were used in the algorithm. As the result, it was found
that 1,745 of 4,744 word/word groups had more than 50% probability for being in the positive polarity and
2,999 had less than 50% probability for being in the positive polarity. In Table 5, a part of the final version of
the Sentiment Database in Turkish which includes the probability for being in the positive polarity can be seen.
(4) Assigning to polarities with Naive Bayes Algorithm.
Classification is a text mining task based on prediction. Text is incorporated, through prediction, into one
class that has been determined in line with its properties. The procedure of assigning a text to one of the
positive or negative polarities is also examined within the framework of text classification subject in text
mining. Many algorithms have been developed for classification (Naive Bayes, support vector machine, rocchio,
decision trees). In this study, Naive Bayes Algorithm which was simple to use and produced efficient results
was used for the classification of the text data. This algorithm based on the calculation of each criterion’s
effects to the result as probability is among the methods mostly preferred for the class determination in the
cases including two probabilities (positive/negative). The algorithm works according to this principle:
P (text-positive) = P (text-negative) =1/2;

AN OPINION MINING TASK IN TURKISH LANGUAGE
192
Probability of the text for being in the positive polarity = ½ × Multiplication of the probability of each
word/word group that is present both in the text and in the Sentiment Database in Turkish for being in the
positive polarity;
Probability of the text for being in the negative polarity = ½ × Multiplication of the probability of each
word/word group that is present both in the text and in the Sentiment Database in Turkish for being in the
negative polarity.
Table 5
A Part of the Sentiment Database in Turkish including the Probability for Being in the Positive Polarity
Sentiment words Frequencies Hexa code Fill color Probability for being in the positive polarity
en iyi 6,433 FFFF4F
99.99
hoş 76,610 FFF700
92.05474706
kısmetli 5,716 FED73C
89.1455142
katkısız 3,098 FFD93C
89.10427035
büyük 17,058 FFD801
88.80089123
gururlu 3,604 FFB300
82.09990792
dayanıklı 4,128 FFB200
82.09990792
daha çok 128,155 FFA401
79.66678813
belirli 3,826 FE9901
77.28985031
cümbüş 1,223 FEA740
75.97838179
başarılmış 5,312 FEA53F
75.11009634
bütün 8,151 FE992D
74.10444177
bahtiyar 17,849 FF7F00
72.19642672
The greater of the statements will indicate the class of the text. The steps of the procedure of assigning to
polarities with Naive Bayes Algorithm can be summarized in this way:
Since Text Mining Practices require working only on text, firstly, the post/comment data are cleaned from
punctuations and numbers, all data are written in lower case letters;
The cleaned texts are spell checked. This spell check can be performed using various programs. For
example, the error correction module in Microsoft Word text editor can be used;
Each word in the post/comment that will be analyzed is compared to the word/word group recorded in the
Sentiment Database in Turkish and bit weighting is performed; for the words that are found value 1 and for the
words that are not found value 0 is given. The repetitions in the text are not considered (The algorithms in
which the repetition numbers are used work with different principles. For this reason, these algorithms were not
used although they were in the English training set). At this point, it should be useful to refer to the task of
finding stems: Some words in the post/comments may not be found in the Sentiment Database in Turkish
because they have inflectional suffixes. However, this disadvantage could be resolved through an application
directed at finding the stems of the words. But since the words in the Sentiment Database in Turkish are
adjective and adverb based words, there is no need to find the stems of these words. Because when adjectives
qualify or determine a noun, that is, when they are used as adjectives, cannot take the noun inflectional suffixes
(case suffixes, possessive suffixes and plural suffixes) that they can take when they are used alone, that is,
when they are considered as nouns (“Adjectives”, 2007). Adverbs, on the other hand, are non-finite words.
They do not take noun inflectional (case, possessive, plural suffixes etc.) suffixes. However, the adverbs that
can be used as nouns can take these suffixes when they are used as nouns (“Adverbs”, 2007). For this reason,

AN OPINION MINING TASK IN TURKISH LANGUAGE
193
an application regarding finding stems were not developed;
In order to determine on which polarity is the weighted text vector, the probabilities of “1” cases are taken
into consideration and two values are calculated. These values indicate the probability for being in
positive/negative polarities. As the result of the comparison, post/comment is assigned to the polarity whose
value is greater.
In addition, the length of the posts/comments in terms of word number was looked and the effect of the
length to the success of polarity assignment was investigated in the study.
Using the Model
In order to use this model that will assign the post/comments to positive or negative polarities, a program
that will automatically perform these procedures was developed. For this program called “Opinion Polarity
Detection”, Visual Basic Language was used in Microsoft NET Framework environment and Microsoft SQL
Server 2005 was used as database. Post/comment entry in the program interface is performed through these
steps:
Posts/comments can be manually entered or copied;
The text, if required, can be spell checked. In this case, program gets into contact with the error correction
module of the Microsoft Word text editor and offers alternatives regarding spell check;
Through this procedure, text can be assigned to positive or negative polarity with the help of the
commands operating in the background or if none of the words is included in the Sentiment Database in
Turkish, the words might not be classified.
Findings and Evaluation
(1) Findings Obtained with the Positive and Negative Precision Measures.
In order to test the success of the established model, a test database comprising of 350 positive and 350
negative samples was formed. A sample of the test database can be seen in Table 6.
Table 6
A Sample of the Test Database
Product/Service (Brand) View URL Post/Comment
X Positive http://adimsoyadim.blogspot.com/ I am a positive comment
Y Negative http://adimsoyadim.blogcu.com/ I am a negative comment
The posts/comments obtained in this way were divided into two classes as the posts/comments in the
positive polarity and in the negative polarity for test purposes and their contents were copied to the files named
“pozitifyorum.txt” and “negatifyorum.txt”. For the automatic process of the posts/comments in two text files by
the program, related combo options were developed. In this way, the posts/comments whose polarities are
known can be read from the related text files and whole analyses can be performed. The program, depending on
the hardware properties of the computer used, can perform the test of one word approximately in 0,002277
second. Both analyses are given in Table 7.
In this study, the model’s success of assigning to the related polarity was evaluated with Precision
Measure. Precision Measure is among the frequently used methods in measuring text classification activities
and can be stated as follows:
Precision = Number of the Posts/Comments classified as True/Total Number of Posts/Comments

AN OPINION MINING TASK IN TURKISH LANGUAGE
194
Table 7
Results of the Assignment to the Related Polarity Obtained With the Test Database
Polarity Total
number of
posts/comments Number of th
e posts/comments
classified as true Number
of the posts/comments
classified as false
number of the
posts/comments that
cannot be classified
Positive 350 253 93 4
Negative 350 256 90 4
Positive precision measure = 0.7228 = 72.28%/Negative precision measure = 0.7314 = 73.14%.
At this point, the method regarding the model assigning the positive/negative pole was compared with the
Turney method that is a method about which it being close to this method within the frame of some certain
criteria can be said and their similarities/differences were put forth (Turney, 2002).
Similarities:
 Both methods try to determine the positive/negative pole of the text and are examined within the context
of the opinion mining task;
 Both methods are used in classifying the customer comments as “suggested ones/positive” or
“non-suggested ones/negative”;
 Both methods were picked randomly from the domains where comments were;
 Both methods focus on the semantic orientation of the adjective/adverb based words.
Differences:
 Turney Method is based upon the English language, yet contrary to this, the model mentioned in the study
is based on Turkish language;
 Turney Method functions according to the unsupervised learning basis, the model mentioned in the study
functions accordingly with the semi-supervised learning basis;
 In the Turney Method, semantic orientation of a phrase is calculated as mutual information in between the
phrase given with the words excellent/poor. And in the model mentioned in the study, words are arrayed
according to the distance between the color codes and in this direction they possess the possibility of being at
the positive/negative pole;
 Turney Method’s number of sample comments is 410; study model’s number of sample comments is 700;
 Database of the Turney Method comes from four different domains (automobiles, banks, movies and
travel destinations) that are taken from the http://www.epinions.com. Comments that partake in this site present
the objective views of real persons. And in the test database of the study model, comments come from a lot of
different blogs. Comments that partake in the blogs present the objective views of real persons. Also here, the
comment contents belong to the products/services titled as “white goods, built-in products, electronics, small
home appliances and heaters-air conditioners”;
 For the comments in the Turney method come from four different domains, accuracy values were
calculated separately. These values respectively are 84%, 80%, 65.83%, and 70.53%. Average accuracy value is
74.39%. And the average accuracy value of the study model is 72.71%.
(2) Descriptive Statistics Regarding the Word Number.
In this section, the descriptive statistics regarding the word number of the posts/comments were included
and the effect of word number to the polarity assignment success was investigated. In order to perform this
research, a module that counted the words of the posts/comments was added to the program. The data regarding

AN OPINION MINING TASK IN TURKISH LANGUAGE
195
the column headings such as “post/comment”, “positive value”, “negative value”, “result”, and “word number”
can be listed using the database button in the interface. In Table 8, a sample of the test database accessed with
the database button can be seen. The columns regarding the results and word numbers for each polarity were
copied to Microsoft Excel and the evaluations below were made.
Table 8
A Sample of the Test Database Accessed With the Database Button
Post/Comment Positive value Negative value Result Word number
Comment 1 3.41284223E-10 4.48293103037529E-07 Negative 185
Comment 2 0.008944502 3.07589510232533E-05 Positive 30
Comment 3 0.00147905317 6.00719609260089E-05 Positive 51
Comment 4 0.00393425 0.0212485840062161 Negative 22
While the average word number of the negative posts/comments in the test database is 86.55, the average
word number for the positive entries is 29.66. From this, it can be inferred that contents are expressed with
shorter entries while discontents and complaints are expressed with longer entries.
The average word number for the positive posts/comments in the test database is 29.66. In the obtained
results, the average word number for the entries classified as true is 27.45 while the average word number for
the entries classified as false is 36.15. From this, for the positive polarity it might be stated that the entries
below the average word number are evaluated as true while the entries above the average word number are
evaluated as false.
The average word number for the negative posts/comments in the test database is 86.55. In the obtained
results, the average word number for the entries classified as true is 93.41 while the average word number for
the entries classified as false is 70.41. From this, for the negative polarity it might be stated that the entries
below the average word number are evaluated as false while the entries above the average word number are
evaluated as true.
When looked to the standard deviations:
For the positive polarity, the standard deviation of the entries classified as true is 22.04 and the standard
deviation of the entries classified as false is 36.30. The standard deviation of the 350 entries in this group is
26.82. From this, for the positive polarity, it might be argued that the entries in which the standard deviation is
lesser, that is, the distribution is more balanced in terms of length, are evaluated as true.
For the negative polarity, the standard deviation of the entries classified as true is 143.46 and the standard
deviation of the entries classified as false is 55.88. The standard deviation of the 350 entries in this group is
126.79. From this, for the negative polarity it might be stated that the entries in which the standard deviation is
greater, that is, the distribution is more unbalanced (including extreme values), are evaluated as true. As a
matter of fact, the values in this interval are between extreme points such as 2-1774.
(3) Some Factors Causing Incorrect Results.
The difficulties encountered in opinion mining field are mostly related to spell check. The program
performs the spell check using Microsoft Word 2007 and offers alternatives for spell check if there is any. Spell
check generally can be performed with the related alternative. However, Microsoft Word does not perform a
semantic spell check, which is another text mining task. The cases in which the words are not underlined
although they are misspelled can be given as an example. For example, the word “sakın” in the data collection

AN OPINION MINING TASK IN TURKISH LANGUAGE
196
form might be copied as “sakin” by mistake (probably due to Turkish character problems). Since the word
“sakin” is also a Turkish word, it will not be underlined and it will be accepted as a word which has been
spelled right until the user realizes and corrects is manually. If these words which are assumed to be spelled
right are included in the Sentiment Database in Turkish, the program will find this misspelled word and give it
a probability value. This, in turn, will affect the classification and might cause a wrong evaluation of the
polarities.
On the other hand, the deletion of the numbers and punctuations in the text due to the nature of text mining
might cause other difficulties. For example, the statements “1.” or “2.” in the text are deleted in the first step.
However, these two adjectives are included in the Sentiment Database in Turkish. If these statements written
with numbers and spots were written with alphabetic characters, they would be in the Sentiment Database in
Turkish.
The “ ’ “ character is a frequently used character for adding suffixes to the brand names in the
posts/comments. However, since this character is also considered to be a punctuation mark, it can be deleted in
the first step. For example, “x’tan çok memnunum” might be replaced by “x tan çok memnunum”. In this case,
the suffix “tan (twilight)” will be considered to be a different word and will be searched in the Sentiment
Database in Turkish. This word meaning “seher, alacakaranlık” is included in the database. Therefore, it will be
analyzed and it might cause a wrong evaluation of the polarities. In order to prevent this error, the program was
configured. The suffix is united with the word and becomes “xtan” when there is only an apostrophe. In this
way, the evaluation of the suffix as a separate word is prevented.
The developers of the We Feel Fine website, Kamvar and Harris, state that “positive feelings tend to not
co-occur often with negative feelings, with a few exceptions” (Kamvar & Harris, 2011). The following
word/word groups, with their probabilities for being in the positive class, can be given as examples of these
exceptional words: Obsessed (92.05%), damaged (75.11%), seedy (75.11%), failing (74.93%), wild (74.51%),
cruel (74.1%), unhealthy (74.04%), rebel (72.20%), and unappreciated (69.49%). The Turkish correspondences
of these words might cause that the texts in which they are included are assigned to wrong polarities.
Conclusions
In today’s world, there are many blogs on the internet that keep customer opinions on products and
services and the number of these blogs increases day by day. These blogs, by their conveying nature, offer text
data that will benefit the enterprises in terms of customer orientation in the area of Customer Relationship
Management. Besides, while the data on the enterprise’s database are only accessible by the enterprise, these
blogs are open to anyone. Therefore, positive or negative opinions can be seen by anyone who reads that blog.
For this reason, enterprises have to follow and analyze these blogs in order to protect their reputations. In this
way, enterprises can develop related strategies regarding the decision processes and ensure the increase in
efficiency.
In this regard, this study uses the Opinion Mining mMethods and proposes a model that will assign the
text-based opinion data in Turkish blogs to positive and negative polarities in order to present a general view on
products and services. In order to use this model that will assign the posts/comments to the polarities, a
program that will automatically perform these procedures was developed.
The model’s success of polarity assignment was evaluated with Precision Measure. The Positive Precision
Measure was calculated to be 72.28% and the Negative Precision Measure was calculated to be 73.14%. These

AN OPINION MINING TASK IN TURKISH LANGUAGE
197
obtained results support the validity of the methodology and give hope for the model’s future. A test database
that will be developed by taking more samples will provide for the possible predictions and controls and better
results will be obtained. It is my wish that this study will contribute to the beginning of the “Opinion Mining
Studies in Turkish Language”.
When it comes to the difficulties encountered in the test level, it might be stated that most of these
difficulties were in the area of spell check. As matter of fact, a semantic spell check is required, which is
another text mining task. On the other hand, the deletion of the numbers and punctuations in the text due to the
nature of text mining might cause other difficulties and might lead to wrong polarity evaluations. Besides, the
developers of the We Feel Fine website, of which the sentiment words were used in the training set, state that
positive feelings tend to not co-occur often with negative feelings, however, this might have a few exceptions.
As a matter of fact, these exceptional words might cause that the texts in which they are included are assigned
to wrong polarities.
This classification model, which might be considered as “Listening to the Social Media” in a sense, is also
important in terms of showing the new dimension added to the enterprise-customer interaction. For this reason,
enterprises have to pay attention to the voice of the masses in order to develop their strategies in the related
area.
References
Abbasi, A., CHEN, H., & Salem, A. (2008). Sentiment analysis in multiple languages: Feature selection for opinion classification
in web forums (ACM Trans.). Information Systems, 26(3).
Adjectives. (2007). Source website for Turkish language and literature. Retrieved February 28, 2013, from
http://www.turkceciler.com/sozcuk_turleri/sifatlar.html
Adverbs. (2007). Source Website for Turkish Language and Literature. Retrieved February 28, 2013, from
http://www.turkceciler.com/sozcuk_turleri/zarflar.html
CHEN, B., ZHU, L. L., Kifer, D., & Lee, D. (2010). What is an opinion about? Exploring political standpoints using opinion
scoring model (In Proceedings of The Twenty-Fourth AAAI Conference On Artificial Intelligence (AAAI), Atlanta, GA., July
11-15, 2010, p. 1).
Conrad, J., & Schilder, F. (2007). Opinion mining in legal blogs. In Proceedings of the International Conference on Artificial
Intelligence and Law (ICAIL). New York, N. Y., USA: ACM, 2007, pp. 231-236.
Dave, K., Lawrence, S., & Pennock, D. (2003). Mining the peanut gallery: Opinion extraction and semantic classification of
product reviews. In www ’03: Proceedings of the 12th International Conference on World Wide Web. New York, N. Y.,
USA, ACM, 2003, pp. 519-528.
Devitt, A., & Ahmad, K. (2007). Sentiment polarity identification in financial news: A cohesion-based approach. In Proceedings
of the 45th Annual Meeting of the Association of Computational Linguistics, Association for Computational Linguistics.
Prague, Czech Republic, June 2007, pp. 984-991.
LIU, B. (2013). Opinion mining. Retrieved February 28, 2013, from http://www.cs.uic.edu/~liub/fbs/opinion-mining.pdf
Esuli, A., & Sebastiani, F. S. (2005). Determining the semantic orientation of terms through gloss analysis. In Proceedings of
CIKM-05, 14th ACM International Conference on Information and Knowledge Management. Bremen, DE., 2005, pp.
617-624.
Esuli, A., & Sebastiani, F. S. (2006). A publicly available lexical resource for opinion mining. In Proceedings of Language
Resources and Evaluation (LREC). Italy, Genoa, May 24-26, 2006, pp. 417-422.
Falcon, J. (2010, August 19). Opinion mining in ediscovery. Retrieved February 28, 2013, from
http://jadefalconit.com/opinion-mining/opinion-mining-in-ediscovery
How to Calculate a Complementary Colour (Inc. RGB/HSL Conversion). Retrieved February 28, 2013, from
http://serennu.com/colour/rgbtohsl.php
Kamvar, S., & Harris, J. (2011). We feel fine and searching the emotional web, WSDM’11. In Proceedings of the Fourth ACM
International Conference on Web Search and Data Mining. Hong Kong, February 9-12, 2011, pp. 117-126.

AN OPINION MINING TASK IN TURKISH LANGUAGE
198
Kamps, J., Marx, M., Mokken, R. J., & De Rijke, M. (2004). Using word net to measure semantic orientation of adjectives. In
Proceedings of LREC-04, 4th International Conference on Language Resources and Evaluation. Volume IV, Lisbon PT,
2004, pp. 1115-1118.
Kurt, H. (1998). Grammar for elementary school 7th grade. İstanbul, Morpa Kültür Yayınları.
Levene, M. (2010). An introduction to search engines and web navigation (2 nd.). New Jersey, Wiley Publisher.
LIU, B. (2007a). Web data mining. Chicago, Springer.
LIU, B. (2007b). From web content mining to natural language processing. ACL-2007 Tutorial, Prague. Retrieved February 28,
2013, from http://www.cs.uic.edu/~liub/acl-07-tutorial-wcm-to-nlp.pdf
Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis, foundation, and trends. Information Retrieval, 2(1-2).
QIU, G., LIU, B., BU, J. J., & CHEN, C. (2010). Opinion word expansion and target extraction through double propagation.
Computational Linguistics, 1(1), 1-18.
RGB Color Space. (2013, February 13). Wikipedia. Retrieved February 28, 2013, from
http://tr.wikipedia.org/wiki/RGB_renk_uzay%C4%B1
Sentiment Analysis. (2013, February 28). Wikipedia. Retrieved from http://en.wikipedia.org/wiki/sentiment_analysis
Stavrianou, A., & Chauchat, J. H. (2008). Opinion mining issues and agreement identification in forum texts (FODOP 2008,
Atelier Fouille Des Données D’opinions, Fontainebleau, France, 27 May 2008, pp. 51-58).
Turney, P. (2002). Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In
Proceedings of the 40th Annual Meeting On Association For Computational Linguistics (ACL ’02). Morristown, N. J., USA,
July 2002, pp. 417-424).
Verbeke, M., & Eynde, W. (2013). Opinion mining. Retrieved February 28, 2013, from
http://people.cs.kuleuven.be/~bettina.berendt/webmining10/l3.pdf
XU, F. Y., & CHENG, X. W. (2011, January 19). Opinion mining. Retrieved June 7, 2011, from
http://ebookbrowse.com/opinion-mining-2011-lecture-pdf-d90656735