Content uploaded by David E. Losada
Author content
All content in this area was uploaded by David E. Losada on Feb 16, 2014
Content may be subject to copyright.
A preview of the PDF is not available
Statistical query expansion for sentence retrieval and its effects on weak and strong queries
Abstract and Figures
The retrieval of sentences that are relevant to a given information need is a challenging passage retrieval task. In this
context, the well-known vocabulary mismatch problem arises severely because of the fine granularity of the task. Short queries,
which are usually the rule rather than the exception, aggravate the problem. Consequently, effective sentence retrieval methods
tend to apply some form of query expansion, usually based on pseudo-relevance feedback. Nevertheless, there are no extensive
studies comparing different statistical expansion strategies for sentence retrieval. In this work we study thoroughly the
effect of distinct statistical expansion methods on sentence retrieval. We start from a set of retrieved documents in which
relevant sentences have to be found. In our experiments different term selection strategies are evaluated and we provide empirical
evidence to show that expansion before sentence retrieval yields competitive performance. This is particularly novel because expansion for sentence retrieval is often done after sentence retrieval (i.e. expansion terms are mined from a ranked set of sentences) and there are no comparative results available between both
types of expansion. Furthermore, this comparison is particularly valuable because there are important implications in time
efficiency. We also carefully analyze expansion on weak and strong queries and demonstrate clearly that expanding queries
before sentence retrieval is not only more convenient for efficiency purposes, but also more effective when handling poor
queries.
Figures - uploaded by David E. Losada
Author content
All figure content in this area was uploaded by David E. Losada
Content may be subject to copyright.
... Ko et al. [34] performed query expansion from assumed relevant sentences to generate snippet. Bando et al. [36] and Losada [35] in general used the same technique but obtained expansion terms from initially retrieved top documents [36]. Takamura and Okumura [30] formulated a summarization task as a maximum coverage problem. ...
... This is a novel summary that was generated using query expansion from related YA answers by applying LCA (Local Context Analysis) technique [35], [36], [47]. None of previous work has applied query expansion using external CQA resources for document summarization. ...
... We set N equals to 45 following the best setting of Bando et al [36]. Following [35], [36], the sentences are then ranked according to their similarity with respect to the expanded query using the sentence ranking method by Allan et al [48]. ...
... Chúng được rút trích từ các nguồn được biên tập cẩn thận bởi con người. Nhóm một bao gồm các phương pháp như: (1) phản hồi sự liên quan (relevance feedback) [61], [36]; (2) phản hồi sự liên quan giả (pseudo-relevance feedback) [52], [48]; (3) sử dụng nhật kí người dùng [77], [14]; và (4) sử dụng sự đồng xuất hiện [53], [33]. ...
... Hơn nữa, công trình này dành cho hệ thống hỏi-đáp chứ không dành cho truy hồi tài liệu. Trong [48], hệ thống kết hợp giải thuật phản hồi liên quan giả với kĩ thuật phân tích nội dung cục bộ để mở rộng truy vấn. ...
Currently, the text document retrieval systems have many challenges in exploring the semantics of queries and documents. Each query implies information which does not appear in the query but the documents related with the information are also expected by user. The disadvantage of the previous spreading activation algorithms could be many irrelevant concepts added to the query. In this paper, a proposed novel algorithm is only activate and add to the query named entities which are related with original entities in the query and explicit relations in the query.
... With rapid development of the World Wide Web and e-societies, Information Retrieval (IR) has many challenges in discovering and exploiting those rich and huge information resources. Semantic search improves search precision and recall by understanding user's intent and the contextual meaning of concepts in documents and queries (Huston and Croft, 2010;Losada, et al, 2010;Egozi, et al, 2011). ...
... In Wang and Zhai (2008), the authors exploit synonyms or co-occurring relations in search engine logs for repairing or expanding queries. In Losada, et al. (2010), the system uses pseudo-relevance feedback to expand queries. However, the two systems do not take account relations in a query. ...
Named entities and WordNet words are im-portant in defining the content of a text in which they occur. Named entities have onto-logical features, namely, their aliases, classes, and identifiers. WordNet words also have ontological features, namely, their synonyms, hypernyms, hyponyms, and senses. Those features of concepts may be hidden from their textual appearance. Besides, there are related concepts that do not appear in a query, but can bring out the meaning of the query if they are added. The traditional constrained spreading activation algorithms use all relations of a node in the network that will add unsuitable information into the query. Meanwhile, we only use relations represented in the query. We propose an ontology-based generalized Vector Space Model to semantic text search. It discovers relevant latent concepts in a query by relation constrained spreading activation. Besides, to represent a word having more than one possible direct sense, it combines the most specific common hypernym of the remaining undisambiguated multi-senses with the form of the word. Experiments on a benchmark dataset in terms of the MAP measure for the retrieval performance show that our model is 41.9% and 29.3% better than the purely keyword-based model and the traditional constrained spreading activation model, respectively.
... Since there are many users sharing their opinions and experiences via social media, there are limitations as viewpoints subject to change with time. Losada et al., [14] aim to make a detailed work on the outcomes of the strategies used for query expansion for retrieving sentences and also present's an excellent statistical query expansion method. The experiment assesses different term selection methods and provide a fact-based documentation to show that performance that can be improved if expansion is done before sentence retrieval. ...
In today’s world, due to the steep rise in internet users, Community Question Answering (CQA) has attracted many research communities. In order to provide the correct and perfect answer to the user asked question from a given large collection of text data, understanding the question properly to suggest a precise answer is a challenging task. Therefore, Question Answering (QA) system is a challenging task than a common information retrieval task done by many search engines. In this paper, an automatic prediction of the quality of CQA answers is proposed. This is accomplished by using five well known machine learning algorithms. Usually, questions asked by the user are based on a topic or theme. We try to exploit this feature in our work by identifying the category of the question posted and further map with the corresponding question. Similarly, for the answers posted by the multiple user’s are processed as answer for category mapping. Here, the results show that for Question Classification (QA), Linear Support Vector Classification (LSVC) is found to be the best classifier and Multinomial Logistic Regression (MLR) is the most suitable for Answer Classification (AC). The MS Macro dataset is used as the underlying dataset for retrieving and testing the question and answer classifiers. The Yahoo Answers are used as a golden reference during the testing throughout our experiments. Experiments results show that the proposed technique is efficient and outperforms Metzler and Kanungo’s (MK++) [1] while providing the best answer summary satisfying the user’s queries.
... One of the first and most successful methods for sentence retrieval is the TF-ISF defined as [5,4,23]: ...
Sentence retrieval is an information retrieval technique that aims to find sentences corresponding to an information need. It is used for tasks like question answering (QA) or novelty detection. Since it is similar to document retrieval but with a smaller unit of retrieval, methods for document retrieval are also used for sentence retrieval like term frequency—inverse document frequency (TF-IDF), BM25, and language modeling-based methods. The effect of partial matching of words to sentence retrieval is an issue that has not been analyzed. We think that there is a substantial potential for the improvement of sentence retrieval methods if we consider this approach. We adapted TF-ISF, BM25, and language modeling-based methods to test the partial matching of terms through combining sentence retrieval with sequence similarity, which allows matching of words that are similar but not identical. All tests were conducted using data from the novelty tracks of the Text Retrieval Conference (TREC). The scope of this paper was to find out if such approach is generally beneficial to sentence retrieval. However, we did not examine in depth how partial matching helps or hinders the finding of relevant sentences.
... Next, the most similar extracted sentences to the original content of metadata units are added. The reason behind using sentences instead of terms is due to two underlying merits: Firstly, it introduces more context to the query than term-based expansion [17]. And secondly a sentence represents a more natural semantic unit of text than terms [18]. ...
Digital Resource Objects (DROs) suffer from
shortage of its contents, and this leads to reduction in the
effectiveness in its retrieval results. In order to increase the
retrieval effectiveness of DRO, adding extra information to its
contents is required. However, the extra information must be
related to the structure of DRO. Each document contains
metadata units with multiple topics. Document expansion (DE)
methods utilize the unstructured documents to increase the
document contents. In the same way, in this paper, an
Enhanced Document Expansion (EDE) method is proposed by
utilizing structured documents. DE method is a way of feeding
and providing documents with new information to increase the
effectiveness of the documents. Usually, traditional DE
methods add terms to the original documents. In the proposed
EDE method, a new procedure to increase the information
content according to specific steps is added and aimed at
adding new information which is more relevant and closer to
each metadata unit in each document. The proposed EDE
method calculates the nearest sentences to the content of the
metadata unit by improving the probability estimation
equation. The experiments which are conducted on cultural
heritage CHiC2013 collections show a statistically significant
improvement over the traditional document expansion
methods.
... For example, if a sentence that obtains a high score in measuring the similarity between a sentence and the query is likely to include an answer to the question, then a related sentence, which may not be similar to the input query itself, is also likely to include an answer. This idea is modelled by the following combination model (Losada, 2010;Miao et al., 2010;Zhao et al., 2009): ...
Question answering system aims at retrieving precise information from a large collection of documents. This work presents a question answering method to apply on Hadith in order to provide an informative answer corresponding to the user's query. Hadith englobes stories and qualification of the prophet Muhammad (PBSL). It also includes the sayings of his companions and their disciples.
The problem with current methods is that they fail to capture the meaning when comparing a sentence and a user's query; hence there is often a conflict between the extracted sentences and user's requirements. However, our proposed method has successfully tackled this problem through: (1) avoiding extract a passage whose similarity with the query is high but whose meaning is different. (2) Computing the semantic and syntactic similarity of the sentence-to-sentence and sentence-to-query. (3) Expanding the words in both the query and sentences to tackle the fundamental problem of term mismatch between sentences and the user's query. Furthermore, in order to reduce redundant Hadith texts, the proposed method uses the greedy algorithm to impose diversity penalty on the sentences. The experimental results display that the proposed method is able to improve performance compared with the existing methods on Hadith datasets.
... An extension of tf.idf weighting method using a fuzzy based model was proposed by Holi, Hyvonen, & Lindgren (2006) and they observed that weighting method produced ranking that matched human judgment. (Allan, Wade & Bolivar, 2003;Losada, 2010) applied a Vector Space Model of sentence level retrieval using the variant of tf.idf as tfisf 2 , where weight is applied at the sentence level and performed much better. Long queries formed after expansion may have greater chances of query-drift due to presence of drifting words. ...
Query Expansion is the process of selecting relevant words that are closest in meaning and context to that of the keyword(s) of query. In this paper, a statistical method of automatically selecting contextually related words for expansion, after identifying a pattern in their score, is proposed. Words appearing in top 10 relevant document is given a score w.r.t partitions they appear in. Proposed statistical method, identifies a pattern of central tendency in the high scores and selects the right group of words for query expansion. The objective of the method is to keep the expanded query with minimum words (light), and still give statistically significant MAP values compared to the original query. Experimental results show 17-21% improvement of MAP over the original unexpanded query as baseline but achieves a performance similar to that of the state of the art query expansion models - Bo1 and KL. FIRE 2011 Adhoc English and Hindi data for 50 topics each were used for experiments with Terrier as the Retrieval Engine.
We present a neural architecture for cross-lingual mate sentence retrieval which encodes sentences in a joint multilingual space and learns to distinguish true translation pairs from semantically related sentences across languages. The proposed model combines a recurrent sequence encoder with a bidirectional attention layer and an intra-sentence attention mechanism. This way the final fixed-size sentence representations in each training sentence pair depend on the selection of contextualized token representations from the other sentence. The representations of both sentences are then combined using the bilinear product function to predict the relevance score. We show that, coupled with a shared multilingual word embedding space, the proposed model strongly outperforms unsupervised cross-lingual ranking functions, and that further boosts can be achieved by combining the two approaches. Most importantly, we demonstrate the model’s effectiveness in zero-shot language transfer settings: our multilingual framework boosts cross-lingual sentence retrieval performance for unseen language pairs without any training examples. This enables robust cross-lingual sentence retrieval also for pairs of resource-lean languages, without any parallel data.
The ranking of search results largely determines the quality of service (QoS) of a meta-search engine (MSE). To address the demand of big data applications, this paper proposes a new method considering factors such as network bandwidth, client and limit server resources. In this method, Web pages with the same contents (but with different URLs) are identified by calculating similarity among contents of pages traversed by the user and those of pages not yet traversed. Hence, deviation of statistics about the user’s intent for traversing caused by factors such as ranking differences in the orders of traversing and repeated contents of Web pages can be eliminated. While a search service is being provided, each component search engine (CSE) weight can be given dynamically before returned results receive a second rotary ranking in combination with initial ranking information. Experimental results and statistics show that (1) the numbers of traversals and downloads can be decreased; (2) the ratio of the number of pages clicked by the user to that of pages navigated can also be decreased; (3) the matching degree between searches/traversals and returned results can be increased; and (4) the stability of a search engine can be improved by taking into account the factor of repeated contents of Web pages.
Réflexions sur la surcharge cognitive.
Cet article aborde le problème de la saturation cognitive tel que les individus le vivent au quotidien sur leur lieu de travail. Il examine d'abord une série d'hypothèses sur les causes de ce phénomène : trop d'information en "push" et en "pull", le multi-tâche et les interruptions, un environnement de travail mal conçu. En passant, il pose une série de questions : la mesure de la qualité de l'information, la forme de sa fonction d'utilité, la pertinence de différentes stratégies de gestion des flux. Ensuite, il cherche à bâtir un cadre d'analyse pour améliorer la conception des environnements de travail. Il enrichit la formalisation de Simon et Newell sur l'espace de la tâche, en montrant que les stratégies cognitives du sujet peuvent l'amener à des détours qui, sans être des tâches proprement dites, sont des reformulations du problème qui facilitent sa résolution. Cette avancée permet de poser plus clairement la question de la charge cognitive et des coûts cognitifs qui découlent de l'environnement. L'article distingue notamment la charge de calcul, la mémoire, la concentration, le stress. Il montre sur quelques exemples simples comment un réaménagement de l'environnement peut abaisser la structure de coûts pour l'opérateur, et lui procurer une aide dans son travail cognitif .
In this paper, we examine a number of newly applied methods for combining pre-retrieval query performance predictors in order
to obtain a better prediction of the query’s performance. However, in order to adequately and appropriately compare such techniques,
we critically examine the current evaluation methodology and show how using linear correlation coefficients (i) do not provide
an intuitive measure indicative of a method’s quality, (ii) can provide a misleading indication of performance, and (iii)
overstate the performance of combined methods. To address this, we extend the current evaluation methodology to include cross
validation, report a more intuitive and descriptive statistic, and apply statistical testing to determine significant differences.
During the course of a comprehensive empirical study over several TREC collections, we evaluate nineteen pre-retrieval predictors
and three combination methods.
In this paper we propose a novel sentence retrieval method based on extracting highly frequent terms from top retrieved documents. We compare it against state of the art sentence retrieval techniques, in- cluding those based on pseudo-relevant feedback, showing that the ap- proach is robust and competitive. Our results reinforce the idea that top retrieved data is a valuable source to enhance retrieval systems. This is especially true for short queries because there are usually few query- sentence matching terms. Moreover, the approach is particularly promis- ing for weak queries. We demonstrate that this novel method is able to improve significantly the precision at top ranks when handling poorly specified information needs.
In a full-text natural-language retrieval system, local feedback is the process of formulating a new improved search based on clustering terms from the documents returned in a previous search of any given query. Experiments were run on a database of US patents. It is concluded that in contrast to global clustering, where the size of matrices limits applications to small databases and improvements are doubtful, local clustering is practical also for large databases and appears to improve overall performance, especially if metrical constraints and weighting by proximity are embedded in the local feedback. The local methods adapt themselves to each individual search and produce useful searchonyms - terms which are ″synonymous″ in the context of one query. Searchonyms lead to new improved search formulations both via manual and via automated feedback.
The issue of query performance prediction has been stud-ied in the context of text retrieval and Web search. In this paper, we investigate this issue in an intranet environment. The collection used is a crawl of the dcs.gla.ac.uk domain, and the queries are logged from the domain search engine, which is powered by the Terrier platform. We propose an automatic evaluation methodology generating the mean av-erage precision of each query by cross-comparing the output of diverse search engines. We measure the correlation of two pre-retrieval predictors with mean average precision, which is obtained by our proposed evaluation methodology. Re-sults show that the predictors are very effective for 1 and 2-term queries, which are the majority of the real queries in the intranet environment.
The detection of new information in a document stream is an important component of many potential applications. In this paper, a new novelty detection approach based on the identification of sentence level patterns is proposed. Given a user's information need, some patterns in sentences such as combinations of query words, named entities and phrases, may contain more important and relevant information than single words. Therefore, the proposed novelty detection approach focuses on the identification of previously unseen query-related patterns in sentences. Specifically, a query is preprocessed and represented with patterns that include both query words and required answer types. These patterns are used to retrieve sentences, which are then determined to be novel if it is likely that a new answer is present. An analysis of patterns in sentences was performed with data from the TREC 2002 novelty track and experiments on novelty detection were carried out on data from the TREC 2003 and 2004 novelty tracks. The experimental results show that the proposed pattern-based approach significantly outperforms all three baselines in terms of precision at top ranks.