Article

The Effects Of Query Complexity, Expansion And Structure On Retrieval Performance In Probabilistic Text Retrieval

Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

ueries using all search facets identified from requests, low complexity was achieved by formulating queries with major facets only. Query expansion was based on a thesaurus, from which the expansion keys were elicited for queries. There were five expansion types: (1) the first query version was an unexpanded, original query with one search key for each search concept (original search concepts) elicited from the test thesaurus; (2) the synonyms of the original search keys were added to the original query; (3) search keys representing the narrower concepts of the original search concepts were added to the original query; (4) search keys representing the associative concepts of the original search concepts were added to the original query; (5) all previous expansion keys were cumulatively added to the original query. Query structure refers to the syntactic structure of a query expression, marked with query operators and parentheses. The structure of queries was either weak (queries with n

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... Strong query structuring has been shown to be beneficial in both monolingual IR (Kekäläinen, 1999) and CLIR (Pirkola, 1998). (Ingwersen and Järvelin, 2005). ...
... A document may discuss a topic with various alternative concepts and synonymic expressions. Therefore, it is hard for users to formulate queries that would cover all of the possible vantage points to the topic (Kekäläinen, 1999). Furthermore, user queries usually consist of but a few words. ...
... Views differ on whether QE actually improves IR performance significantly. Kekäläinen (1999) found that strong query structuring is vital for QE success, especially when a large number of new keys are added. She also found that using the synonym structure of the InQuery language (see Section 2.2) to represent facets is advantageous in QE. ...
... Text document retrieval tasks typically involve large numbers of variables (words or terms). This was obvious also in the Finnish newspaper article collection [15] which we aim to process for information retrieval both with cluster analysis (see, for example [6,11,26]) and instance-based learning techniques [23]. Variable selection is the traditional approach to tackle this problem in the area of information retrieval. ...
... Since the influence of single words is slight in the context of entire documents, the unidentified words had only a marginal effect on the clustering results. Our 53,893 text documents were articles from three Finnish newspapers from the late 1980s to the early 1990s [15]. The articles of the collection include earlier relevance assessments [15,16] concerning 30 queries like: ''The meeting of Presidents George Bush and Mihail Gorbatshov in Helsinki in September 1990, agenda of their negotiations, and decisions and agreements signed''. ...
... Our 53,893 text documents were articles from three Finnish newspapers from the late 1980s to the early 1990s [15]. The articles of the collection include earlier relevance assessments [15,16] concerning 30 queries like: ''The meeting of Presidents George Bush and Mihail Gorbatshov in Helsinki in September 1990, agenda of their negotiations, and decisions and agreements signed''. The four-level graded relevance arrangement was as follows: (0) irrelevant; (1) marginally relevant, tentatively touches on a topic; (2) relevant, adds information about a matter; and (3) very relevant, the headline of an article concerns a topic. ...
Article
Clustering groups document objects represented as vectors. An extensive vector space may cause obstacles to applying these methods. Therefore, the vector space was reduced with principal component analysis (PCA). The conventional cosine measure is not the only choice with PCA, which involves the mean-correction of data. Since mean-correction changes the location of the origin, the angles between the document vectors also change. To avoid this, we used a connection between the cosine measure and the Euclidean distance in association with PCA, and grounded searching on the latter. We applied the single and complete linkage and Ward clustering to Finnish documents utilizing their relevance assessment as a new feature. After the normalization of the data PCA was run and relevant documents were clustered.
... Tutkimme myös relevanssilajittelua ontologioiden ja kyselyjen laajentamisen yhteydessä (Kekäläinen 1999) ja rakenteisten dokumenttien hakumenetelmiä (Arvola, Junkkari & Kekäläinen 2005). ...
Article
Full-text available
Tämä artikkeli on suomenkielinen lyhennelmä Salton-palkinnon (http://sigir.org/awards/gerard-salton-awards/) vastaanottopuheesta, jonka professori (emer.) Kalervo Järvelin piti 41. ACM SIGIR –konferenssin yhteydessä 9. heinäkuuta 2018 Ann Arborissa (MI) USA:ssa. Aluksi tekijä kertoo taustastaan ja tarkastelee sitten joitakin tiedon hankinnan ja haun osa-alueita, jotka ovat olleet keskeisiä tekijän tutkimustyössä. Näihin kuuluvat tehtäväperusteinen tiedon hankinta ja informaatiovuorovaikutus, luonnollisen kielen käsittely yksi- ja monikielistä tiedonhakua varten ja tiedonhaun evaluointimetriikat. Lopuksi tekijä luonnostelee lähestymistavan informaatiovuorovaikutusta koskevan tutkimuksen organisointiin. Puhe on julkaistu englanniksi kokonaan SIGIR Forum verkkolehdessä, vol. 52 no. 2 (http://sigir.org/wp-content/uploads/2019/01/p052.pdf).
... Qiu and Frei [84] and Crouch and Yang [85] presented ways to develop thesaurus automatically from text for query expansion and shows improved retrieval performance. Although Voorhees [86] showed that WordNet is not useful in improving text retrieval, Kekäläinen [87] and Sormunen et al. [88] argued that domain specific thesauri can significantly improve text retrieval. Recently, Clough and Stevenson [89] also showed that using EuroWordNet for cross-language information retrieval is effective. ...
Thesis
In biomedicine, the explosion of textual knowledge sources has introduced formidable challenges for knowledge-aware information systems. Traditional knowledge acquisition methods have been proved costly, resource intensive and time consuming. Automation of large scale knowledge acquisition systems requires narrowing down the semantic gap between biomedical texts and structured representations. In this context, this study proposes a knowledge acquisition framework from biomedical texts. This contributes towards reducing efforts, time and cost incurred to minimaize ontology acquisition bottlenecks.The proposed framework approximates, models, structures and ontologizes implicit knowledge buried in biomedical texts. In the framework, the semantic disambiguator approximates biomedical artefacts from biomedical texts. The conceptual disambiguator models and structures the biomedical knowledge abstracted from the domain texts. Ontologization presents an explicit interpretation of biomedical artefacts and conceptualizations. The components of the framework are instantiated with scientific and clinical text documents and produced about four million concepts and seven million associations. This set of artefacts is structured into the lower ontological knowledge structure where the upper ontology structure is reused from existing ones. The conceptual structure is represented with graph formalism. The formal interpretation is based on OWL DL language primitives and constructs, which generates a set of OWL DL axioms. The set of OWL DL axioms is referred as the OWL ontology (Ko).The extent of approximation and quality of structural design are evaluated using criteria-based methods. A set of metrics is used to measure each criterion and showed encouraging results. Correctness measurements for concept entity are 70% for accuracy, 82% for completeness, 68% for conciseness and 100% for consistency. Quality measurement showed complex ontology structure with metrics values of 986,448 for vocabulary size, 18.73 for connectivity density, 145,246 for tree impurity and 226, 698 for graph entropy. The ontology schema potential metrics values are also 0.80 for relationship richness, 3 for attribute richness and 13,253 for inheritance richness. Ontology clarity showed an average readability, which is 3 attributes on average. The proposed framework has limitations to address the acquisition of individuals and entity attributes, losing cardinality information in the acquisition of the ontological knowledge. These lead to limitations on the formal interpretation of biomedical semantics, which in turn lead to deploy only existential restriction based interpretations. Thus, a way forward has been recommended to enhance semantic disambiguation and ontologization of the proposed framework so that they enable to accommodate the acquisition of cardinality and attribute information.
... The newspapers are Aamulehti (a general newspaper), Keskisuomalainen (a general newspaper) and Kauppalehti (an economics oriented five-day newspaper) and the database consists of 53,893 articles. The articles represent different sections of the newspapers, mostly economics (from all sections of Kauppalehti, some 16,000 articles), and foreign and international affairs (Aamulehti, some 25,000 articles) and articles from all sections of Keskisuomalainen (some 13,000 articles) [31] [32] [33]. ...
Article
Full-text available
Purpose To show that stem generation compares well with lemmatization as a morphological tool for a highly inflectional language for IR purposes in a best‐match retrieval system. Design/methodology/approach Effects of three different morphological methods – lemmatization, stemming and stem production – for Finnish are compared in a probabilistic IR environment (INQUERY). Evaluation is done using a four‐point relevance scale which is partitioned differently in different test settings. Findings Results show that stem production, a lighter method than morphological lemmatization, compares well with lemmatization in a best‐match IR environment. Differences in performance between stem production and lemmatization are small and they are not statistically significant in most of the tested settings. It is also shown that hitherto a rather neglected method of morphological processing for Finnish, stemming, performs reasonably well although the stemmer used – a Porter stemmer implementation – is far from optimal for a morphologically complex language like Finnish. In another series of tests, the effects of compound splitting and derivational expansion of queries are tested. Practical implications Usefulness of morphological lemmatization and stem generation for IR purposes can be estimated with many factors. On the average P‐R level they seem to behave very close to each other in a probabilistic IR system. Thus, the choice of the used method with highly inflectional languages needs to be estimated along other dimensions too. Originality/value Results are achieved using Finnish as an example of a highly inflectional language. The results are of interest for anyone who is interested in processing of morphological variation of a highly inflected language for IR purposes.
... The case demonstrating the effects of multiple degree relevance assessments, and the application of traditional / novel evaluation measures explores query expansion and query structures in probabilistic IR. Kekal~iinen [9], and Kek~ilainen and Jarvelin [11 ] have earlier observed that the structure of queries influences retrieval performance when the number of search keys in queries is high, i.e., when queries are expanded. Query structure refers to the syntactic structure of a query expression, marked with query operators and parentheses. ...
Conference Paper
This paper proposes evaluation methods based on the use of non-dichotomous relevance judgements in IR experiments. It is argued that evaluation methods should credit IR methods for their ability to retrieve highly relevant documents. This is desirable from the user point of view in modern large IR environments. The proposed methods are (1) a novel application of P-R curves and average precision computations based on separate recall bases for documents of different degrees of relevance, and (2) two novel measures computing the cumulative gain the user obtains by examining the retrieval result up to a given ranked position. We then demonstrate the use of these evaluation methods in a case study on the effectiveness of query types, based on combinations of query structures and expansion, in retrieving documents of various degrees of relevance. The test was run with a best match retrieval system (In-Query1) in a text database consisting of newspaper articles. The results indicate that the tested strong query structures are most effective in retrieving highly relevant documents. The differences between the query types are practically essential and statistically significant. More generally, the novel evaluation methods and the case demonstrate that non-dichotomous relevance assessments are applicable in IR experiments, may reveal interesting phenomena, and allow harder testing of IR methods.
... Extracted terms retrieved more useful references partly independently of the search tactics, and partly as used to substitute search terms. This finding is in line with the results from experiments, which show that query expansion based on terms extracted from relevant documents enhance recall (Kekäläinen, 1999) and with observations in Spink and Saracevic (1997), which suggest that in mediated searches terms extracted from retrieved items perform relatively better than terms obtained from users or thesaurus. ...
Article
This article focuses on analysing students' information needs in terms of conceptual understanding of the topic they propose to study and its consequences for the search process and outcome. The research subjects were 22 undergraduates of psychology attending a seminar for preparing a research proposal for a small empirical study. They were asked to make searches in the PsycINFO database for their task in the beginning and end of the seminar. A pre- and postsearch interview was conducted in both sessions. The students were asked to think aloud in the sessions. This was recorded, as were the transaction logs. The results show that during the preparation of research proposals different features of the students' conceptual structure were connected to the search success. Students' ability to cover their conceptual construct by query terms was the major feature affecting search success during the whole process. In the beginning also the number of concepts and the proportion of subconcepts in the construct contributed indirectly via search tactics to retrieving partly useful references. Students' ability to extract new query terms from retrieved items improved search results.
Article
This article is a lightly edited text of the author's Salton Award Keynote: Information Interaction in Context, presented at the 41st SIGIR conference in Ann Arbor, July 9th, 2019. It first gives some personal background and then discusses some important areas of information seeking and IR in the author's research work. These include task-based information behavior and interaction, natural language processing to improve document ranking in mono- and cross-language IR, and IR evaluation metrics. Finally, the article proposes a way to organize research on information interaction.
Article
This paper proposes evaluation methods based on the use of non-dichotomous relevance judgements in IR experiments. It is argued that evaluation methods should credit IR methods for their ability to retrieve highly relevant documents. This is desirable from the user point of view in modem large IR environments. The proposed methods are (1) a novel application of P-R curves and average precision computations based on separate recall bases for documents of different degrees of relevance, and (2) two novel measures computing the cumulative gain the user obtains by examining the retrieval result up to a given ranked position. We then demonstrate the use of these evaluation methods in a case study on the effectiveness of query types, based on combinations of query structures and expansion, in retrieving documents of various degrees of relevance. The test was run with a best match retrieval system (In- Query I) in a text database consisting of newspaper articles. The results indicate that the tested strong query structures are most effective in retrieving highly relevant documents. The differences between the query types are practically essential and statistically significant. More generally, the novel evaluation methods and the case demonstrate that non-dichotomous relevance assessments are applicable in IR experiments, may reveal interesting phenomena, and allow harder testing of IR methods.
Article
Full-text available
Abstract Purpose – The purpose of this article is to discuss advantages and disadvantages of various means to manage morphological variation of keywords in monolingual information retrieval. Design/methodology/approach – The authors present a compilation of query results from 11 mostly European languages and a new general classification of the language dependent techniques for management of morphological variation. Variants of the different techniques are compared in some detail in terms of retrieval effectiveness and other criteria. The paper consists mainly of an overview of different management methods for keyword variation in information retrieval. Typical IR retrieval results of 11 languages and a new classification for keyword management methods are also presented. Findings – The main results of the paper are an overall comparison of reductive and generative keyword management methods in terms of retrieval effectiveness and other broader criteria. Originality/value – The paper is of value to anyone who wants to get an overall picture of keyword management techniques used in IR. Keywords Information retrieval, Language, Knowledge processes Paper type General review
Article
This study explored how the expression of search facets and relevance feedback (RF) by users was related to search success in interactive and automatic query expansion in the course of the search process. Search success was measured both in the number of relevant documents retrieved, whether identified by users or not. Research design consisted of 26 users searching for four TREC topics in Okapi IR system, half of the searchers using interactive and half automatic query expansion based on RF. The search logs were recorded, and the users filled in questionnaires for each topic concerning various features of searching. The results showed that the exhaustivity of the query was the most significant predictor of search success. Interactive expansion led to better search success than automatic expansion if all retrieved relevant items were counted, but there was no difference between the methods if only those items recognised relevant by users were observed. The analysis showed that the difference was facilitated by the liberal relevance criterion used in TREC not favouring highly relevant documents in evaluation.
Article
This article proposes evaluation methods based on the use of nondichotomous relevance judgements in IR experiments. It is argued that evaluation methods should credit IR methods for their ability to retrieve highly relevant documents. This is desirable from the user point of view in modern large IR environments. The proposed methods are (1) a novel application of P-R curves and average precision computations based on separate recall bases for documents of different degrees of relevance, and (2) generalized recall and precision based directly on multiple grade relevance assessments (i.e., not dichotomizing the assessments). We demonstrate the use of the traditional and the novel evaluation measures in a case study on the effectiveness of query types, based on combinations of query structures and expansion, in retrieving documents of various degrees of relevance. The test was run with a best match retrieval system (InQuery1) in a text database consisting of newspaper articles. To gain insight into the retrieval process, one should use both graded relevance assessments and effectiveness measures that enable one to observe the differences, if any, between retrieval methods in retrieving documents of different levels of relevance. In modern times of information overload, one should pay attention, in particular, to the capability of retrieval methods retrieving highly relevant documents.
Conference Paper
The role of subject and search knowledge in query expansion (QE) is an unmapped terrain in research on information retrieval. It is likely that both have an impact on the process and outcome of QE. In this paper our aim is an analytical study of the connections between subject and search knowledge and term selection in QE based both on thesaurus and relevance feedback. We will also argue analytically how thesaurus, term suggestion in interactive QE and term extraction in automatic QE support users with differing levels of subject knowledge in their pursuit of search concepts and terms. It is suggested that in QE the initial query concepts representing the information need should not be treated as separate entities, but as conceptually interrelated. These interrelations contribute to the meaning of the conceptual construct, which the query represents, and this should be reflected in the terms identified for QE.
Conference Paper
Full-text available
A retrospective method for the performance comparison of Boolean and best-match queries is introduced. The method is based on the interactive optimisation of queries by a group of test searchers using a query performance analyser. The case experiment focused on comparing the maximum effectiveness of Boolean exact-match queries, and structured and unstructured best-match queries. The experiment verified the problems in maintaining precision of Boolean queries at high recall levels. Interesting similarities were also observed between structured and unstructured best-match queries giving new light on the results of earlier studies. The case experiment showed that the proposed evaluation method yields more elaborated results in comparisons than earlier query-centred methods.
Conference Paper
Full-text available
In this paper we show that keyword variation of a morphologically complex language, Finnish, can be handled effectively for IR purposes by generating only the textually most frequent forms of the keyword. Theoretically Finnish nouns have about 2,000 different forms, but occurrences of most of the forms are rare. Corpus statistics showed that about 84 – 88 per cent of the occurrences of inflected noun forms are forms of only six cases out of the 14 possible. This number – maximally 2*6 – of keyword’s variant forms makes it feasible to try them all in a search. IR results of the frequent keyword form variation coverage were tested with three to twelve keyword variant forms in two test collections, TUTK and CLEF 2003’s Finnish material. The results show that the frequent keyword form generation method competes well with the gold standard, lemmatization, with nine and twelve variant keyword forms.
Article
The paper reports the development of Dipe-D, a knowledge-based procedure for the formulation of Boolean queries in information retrieval. Dipe-D creates a query in two steps: (1) the user's information need is developed interactively, while identifying the concepts of the information need, and subsequently (2) the collection of concepts identified is automatically transformed into a Boolean query. In the first step, the subject area—as represented in a knowledge base—is explored by the user. He does this by means of specifying the (concepts that meet his) information need in an artificial language and looking through the solution as provided by the computer. The specification language allows one to specify concepts by their features, both in precise terms as well as vaguely. By repeating the process of specifying the information need and exploring the resulting concepts, the user may precisely single out the concepts that describe his information need. In the second step, the program provides the designations (and variants) for the concepts identified, and connects them by appropriate operators. Dipe-D is meant to improve on existing procedures that identify the concepts less systematically, create a query manually, and then sometimes expand that query. Experiments are reported on each of the two steps; they indicate that the first step identifies only but not all the relevant concepts, and the second step performs (at least) as good as human beings do.
Article
We propose a method for performing evaluation of relevance feedback based on simulating real users. The user simulation applies a model defining the user’s relevance threshold to accept individual documents as feedback in a graded relevance environment; user’s patience to browse the initial list of retrieved documents; and his/her effort in providing the feedback. We evaluate the result by using cumulated gain-based evaluation together with freezing all documents seen by the user in order to simulate the point of view of a user who is browsing the documents during the retrieval process. We demonstrate the method by performing a simulation in the laboratory setting and present the “branching” curve sets characteristic for the presented evaluation method. Both the average and topic-by-topic results indicate that if the freezing approach is adopted, giving feedback of mixed quality makes sense for various usage scenarios even though the modeled users prefer finding especially the most relevant documents.
Article
Full-text available
Search facilitated with agglomerative hierarchical clustering methods was studied in a collection of Finnish newspaper articles (N = 53,893). To allow quick experiments, clus- tering was applied to a sample (N = 5,000) that was reduced with principal components analysis. The dendrograms were heuristically cut to find an optimal partition, whose clusters were compared with each of the 30 queries to retrieve the best-matching cluster. The four- level relevance assessment was collapsed into a binary one by (A) considering all the relevant and (B) only the highly relevant documents relevant, respectively. Single linkage (SL) was the worst method. It created many tiny clusters, and, consequently, searches enabled with it had high precision and low recall. The complete linkage (CL), average linkage (AL), and Ward's methods (WM) returned reasonably-sized clusters typically of 18 -32 doc uments. Their recall (A: 27-52%, B: 50-82%) and precision (A: 83-90%, B: 18-21%) was higher than and comparable to those of the SL clusters, respectively. The AL and WM clustering had 1- 8% better effectiveness than nearest neighbor searching (NN), and SL and CL were 1-9% less efficient that NN. However, the differences were statistically insignificant. When evaluated with the liberal assessment A, the results suggest that the AL and WM clustering offer better retrieval ability than NN. Assessment B renders the AL and WM clustering better than NN, when recall is considered more important than precision. The results imply that collections in the highly inflectional and agglutinative languages, such as Finnish, may be clustered as the collections in English, provided that documents are appropriately preprocessed.
Article
Full-text available
A new laboratory-based method for the evaluation of Boolean queries in free-text searching of full-text databases is proposed. The method is based on a controlled formulation of inclusive query plans, on an automatic conversion of query plans into a set of elementary queries, and on composing optimal queries at varying operational levels by combining appropriate sub-sets of elementary queries. The method is based on the idea of reverse engineering, and exploits full relevance data of documents to find the query performing optimally within given operational constraints. The proposed
Article
Full-text available
this paper, the translation polysemy and the dictionary coverage problems were attacked by means of the combination of a general language MRD and a domain specific MR D i.e., a medical dictionary. The domain was restricted to medicine and health by choosing as test requests TREC's (see Harman, 1996) health related topics. The performance of translated Finnish queries against English documents was compared to the performance of original English queries against English documents. Because the domain was medicine and health, it was assumed that the medical dictionary disambiguates word senses, giving less incorrect senses than the general dictionary, and that it contains such search keys that are not found in the general dictionary
Article
In this paper we describe an information retrieval system in which advanced natural language processing techniques are used to enhance the effectiveness of term-based document retrieval. The backbone of our system is a traditional statistical engine that builds inverted index files from pre-processed documents, and then searches and ranks the documents in response to user queries. Natural language processing is used to (a) preprocess the documents in order to extract content-carrying terms, (b) discover inter-term dependencies and build a conceptual hierarchy specific to the database domain, and (c) process the user's natural language requests into effective search queries. During the course of the Text REtrieval Conferences, TREC-1 and TREC-2,∗ our system has evolved from a scaled-up prototype, originally tested on such collections as CACM-3204 and Cranfield, to its present form, which can be effectively used to process hundreds of millions of words of unrestricted text.
Article
The problem of determining an adequate sample size, for an information retrieval experiment comparing two systems on separate samples of requests, is discussed. Minimal sample sizes under different assumptions are derived. It is concluded that such experiments are very likely to require samples of considerable size.
Article
This paper gives a brief description of the Okapi projects and the work of the Centre for Interactive Systems research, as an introduction to this special issue of the Journal of Documentation. Okapi is the name given to an experimental text retrieval system (or rather, family of systems, as will be discussed below), based at City University, London. The current systems and their predecessors have been used as the basis for a series of projects, generally addressing aspects of user information-seeking behaviour and user-system interaction, as well as system design. The projects have been supported extensively by the British Library, and to some degree by a number of other funders. They have been at City since 1989; for the previous seven years they were based at the Polytechnic of Central London (now the University of Westminster). In order to give a picture of the system(s) that now constitute Okapi, it is appropriate to describe one version containing some of the features that have become central to the Okapi projects, and then to indicate the variety of systems now implemented or implementable within the present setup, as well as the directions it may go in the future. In what follows, papers in this issue are referred to by brief titles.
Article
A method of integrating Boolean queries with probabilistic retrieval models is proposed. Boolean queries are interpreted as specifying term dependencies that can be used to correct the document scores obtained with a basic probabilistic model. Alternative methods of obtaining dependency information, such as user-specified phrases, can also be used in this approach. The experimental results indicate that significant performance benefits can be obtained, particularly when dependencies are derived from term phrases identified in natural language queries. © 1986 John Wiley & Sons, Inc.
Article
This paper is intended to serve as a comprehensive introduction to the emerging field concerned with the design and use of ontologies. We observe that disparate backgrounds, languages, tools, and techniques are a major barrier to effective communication among people, organisations, and/or software systems. We show how the development and implementation of an explicit account of a shared understanding (i.e. an `ontology') in a given subject area, can improve such communication, which in turn, can give rise to greater reuse and sharing, inter-operability, and more reliable software. After motivating their need, we clarify just what ontologies are and what purposes they serve. We outline a methodology for developing and evaluating ontologies, first discussing informal techniques, concerning such issues as scoping, handling ambiguity, reaching agreement and producing definitions. We then consider the benefits of and describe, a more formal approach. We re-visit the scoping phase, and discuss the role of formal languages and techniques in the specification, implementation and evaluation of ontologies. Finally, we review the state of the art and practice in this emerging field, considering various case studies, software tools for ontology development, key reearch issues and future prospects.
Article
Introduction This is the first participation of the Graduate School of Library and Information Science, University of California at Los Angeles in the TREC Conference. For TREC--2, Category B, UCLA used a version of the Okapi text retrieval system that was made available to UCLA by CityUniversity, London, UK. OKAPI has been described in TREC1 (Robertson, Walker, Hancock-Beaulieu, Gull & Lau, 1993a) as well as in this conference (Robertson, Walker, Jones, Hancock-Beaulieu, & Gatford, 1994) . Okapi is a simple set-oriented system based on a generalized probabilistic model with facilities for relevance feedback. In addition OKAPI supports a full range of deterministic Boolean and quasi-Boolean operations. 1.1 Objectives The main research objective of the UCLA participation in TREC--2 was to investigate query expansion within the framework as provided byOkapi. More specifically,the objectives were to: ffl use an enhanced version of the Go-See-List (GSL
Meaning and reference
  • H Putnam
Putnam, H. (1973). Meaning and reference. The Journal of Philosophy 70(19), 699–711
Term associations and computational linguistics
  • G Ruge
  • C Schwartz
Ruge, G. & Schwartz, C. (1991). Term associations and computational linguistics. International Classification 18(1), 19- 25
Yleinen kielitiede [General linguistics] Helsinki: Yliopistopaino The use of term position devices in ranked output experiments
  • F Karlsson
Karlsson, F. (1994). Yleinen kielitiede [General linguistics]. Helsinki: Yliopistopaino. [In Finnish.] Keen, E. M. (1991). The use of term position devices in ranked output experiments. Journal of Documentation, 47(1), 1– 22
Classification: Prospects, problems and possibilities Classification research for knowledge representation and organization
  • E Svenonius
Svenonius, E. (1992). Classification: Prospects, problems and possibilities. In N. J. Williamson & M. Hudon (Eds.), Classification research for knowledge representation and organization. Proceedings of 5th international Study Conference on Classification Research. Amsterdam: Elsevier, 5–25
Knowledge organization and terminology: Philosophical and linguistic bases
  • I Dahlberg
Dahlberg, I. (1992). Knowledge organization and terminology: Philosophical and linguistic bases. International Classification, 19(2), 65–71
Aiherelevanssi ja käyttäjärelevanssi tulkinnan näkökulmasta. [Topicality and user-relevance from the point of the view of interpretation.] Kirjastotiede ja informatiikka 14(3), 95–99 Concept-based query expansion in a probabilistic IR system
  • J Kristensen
Kristensen, J. (1995). Aiherelevanssi ja käyttäjärelevanssi tulkinnan näkökulmasta. [Topicality and user-relevance from the point of the view of interpretation.] Kirjastotiede ja informatiikka 14(3), 95–99. [In Finnish.] Kristensen, J. (1996). Concept-based query expansion in a probabilistic IR system. In P. Ingwersen & N. O. Pors (Eds.), Proceedings of the Second International Conference on Conceptions of Library and Information Science: Integration in Perspective. Copenhagen: The Royal School of Librarianship, 281-291
A deductive data model for thesaurus navigation and query expansion Tampere: Informaatiotutkimuksen laitos
  • K Järvelin
  • J Kristensen
  • T Niemi
  • E Sormunen
  • H Keskustalo
Järvelin, K., Kristensen, J., Niemi, T., Sormunen, E. & Keskustalo, H. (1996b). A deductive data model for thesaurus navigation and query expansion. Finnish Information Studies 5. Tampere: Informaatiotutkimuksen laitos, Tampereen yliopisto
Relevance reconsidered ‘96
  • T Saracevic
Saracevic, T. (1996). Relevance reconsidered ‘96. In P. Ingwersen & N. O. Pors (Eds.), Proceedings of the Second International Conference on Conceptions of Library and Information Science: Integration in Perspective. Copenhagen: The Royal School of Librarianship, 201–218
Semantics Cambridge 1 The family name of the present author in the period 1981-1997 was Kristensen. r118 Peat, H The limitations of term co-occurence data for query expansion in document retrieval systems
  • F R J Palmer
  • P Willett
Palmer, F. R. (1982). Semantics. (2nd ed.) Cambridge: Cambridge University Press. 1 The family name of the present author in the period 1981-1997 was Kristensen. r118 Peat, H. J. & Willett, P. (1991). The limitations of term co-occurence data for query expansion in document retrieval systems. Journal of the American Society for Information Science 42(5): 378–383