Automatic Construction of Labled Named Entities for Information Retieval

Abstract

Within the general goal of information retrieval, i.e. finding the documents that are relevant to a user’s information need, there is a problem in terms of the users’ input and search engines’ output. Users submit incomplete and not very informative queries and the search engine returns a list of documents which are considered to be most relevant because only the first few of them are checked by the users. One way of dealing with these problems is to organize documents based on their semantic similarity and allow users to navigate by means of category labels as well as by looking for similar pages. In this study we have tried to harvest labeled clusters of semantically similar named entities which can be used as a first step for web document clustering. We first collect ~44,000 named entities from a thesaurus which is constructed by Dekang Lin applying a word similarity measure based on their distributional pattern. Using their similarity metrics and CLUTO clustering software, we create 2000 semantically similar clusters of the named entities. Then we collect ~305,500 label-instance pairs from the 2007 English Wikipedia dump and implement a labeling algorithm presented by Benjamin Van Durme and M.Pasça (2008) to assign a label to the clusters. This automatic lableing task is able to assign a label which describes the majority of the named entities in 924 of the clusters, which is 46.2% of the total clusters. Finally we evaluate both the clustering and labeling tasks taking 86 randomly selected clusters and on the bases of two native English speaker evaluators’ subjective judgment. According to these evaluators, the clustering task has a purity score of 0.7 and 55% of the labels are acceptable with different degree of accuracy. To check inter-evaluator agreement, we give them 20 similar labeled clusters and they get 0.6 and 0.5 kappa score for clustering and labeling result evaluation.