Elaine Toms

• The University of Sheffield

Serendipity is highly valued as a process for developing original solutions to problems and for innovation. However, it is difficult to capture and thus difficult to measure, but novelty is a key and critical indicator. In this work, we investigate the relationship between user behavioural actions and perceived novelty in the context of browsing. 1...

This book has examined serendipity through a digital information environment lens. It comes at a critical juncture in the development of this relatively nascent research area. Although research papers on serendipity in digital information environments have been bubbling up in conference proceedings and journals across a variety of disciplines for d...

While dichotomized as either a personal aptitude or a phenomenon or event (Merton and Barber, 2004), a review of the serendipity research relating to digital environments reveals that serendipity is primarily approached as either a 1. quality of an event, something, or someone; or a 2. process or experience, which has one or more serendipitous qual...

Given the diversity of the elements contained in serendipity research, serendipity research designs deploy a potpourri of research methods from interviews to experiments, which generate a diverse set of both qualitative and quantitative data for analyzes. Selecting the best methods to understand aspects of serendipity or tools to support serendipit...

Reasons for examining the phenomenon of serendipity have evolved and grown over the past 25 years. Rapid technological change has provided both impetus and inspiration for serendipity research that examines how people adopt, adapt to, use, and, in turn, influence digital environments and how to design them to better support serendipity in the conte...

Evaluating user engagement with search is a critical aspect of understanding how to assess and improve information retrieval systems. While standard techniques for measuring user engagement use questionnaires, these are obtrusive to user interaction, and can only be collected at acceptable intervals. The problem we address is whether there is a les...

Measures of user behaviour and user perception have been used to evaluate interactive information retrieval systems. However, there have been few efforts taken to understand the relationship between these two. In this paper, we investigated both using user actions from log files, and the results of the User Engagement Scale, both of which came from...

For two decades, research has sought to understand serendipity and how it may be facilitated in digital environments such as information visualizations systems, search systems, and social media. The motivation to support serendipity comes from its association with positive outcomes that range from personal benefits to global rewards. To date, resea...

Serendipitous discovery has been a research topic for more than one hundred years, but only recently has it been the focus of attention in library and information science (LIS). In 1960, Bernier published the first article on serendipity in a LIS journal. The number of publications gradually grew until 1990s, after which the field experienced a sig...

The Social Book Search (SBS) Lab investigates book search in scenarios where users search with more than just a query, and look for more than objective metadata. Real-world information needs are generally complex, yet almost all research focuses instead on either relatively simple search based on queries or recommendation based on profiles. The goa...

Users looking for books online are confronted with both professional meta-data and user-generated content. The goal of the Interactive Social Book Search Track was to investigate how users used these two sources of information, when looking for books in a leisure context. To this end participants recruited by four teams performed two different task...

There is broad consensus in the field of IR that search is complex in many use cases and applications, both on theWeb and in domain specific collections, and both in our professional and in our daily life. Yet our understanding of complex search tasks, in comparison to simple look up tasks, is fragmented at best. The workshop addressed many open re...

This research aimed to gain a detailed understanding of how genealogists and historians interact with, and make use of, finding aids in print and digital form. The study uses the lens of human information interaction to investigate finding aid use. Data were collected through a lab-based study of 32 experienced archives' users who completed two tas...

There is broad consensus in the field of IR that search is complex in many use cases and applications, both on the Web and in domain specific collections, and both professionally and in our daily life. Yet our understanding of complex search tasks, in comparison to simple look up tasks, is fragmented at best. The workshop addressed the many open re...

Under what conditions is serendipity most likely to occur? How much is serendipity influenced by what a person brings to the process, and how much by the environment in which the person is immersed? This study assessed (a) selected human characteristics that may influence the ability to experience serendipity (openness to experience, extraversion,...

Serendipity is not an easy word to define. Its meaning has been stretched to apply to experiences ranging from the mundane to the exceptional. Serendipity, however, is consistently associated with unexpected and positive personal, scholarly, scientific, organizational, and societal events and discoveries. Diverse serendipitous experiences share a c...

Purpose – One core element of interactive information retrieval (IIR) experiments is the assignment of search tasks. The purpose of this paper is to provide an analytical review of current practice in developing those search tasks to test, observe or control task complexity and difficulty. Design/methodology/approach – Over 100 prior studies of II...

Personalisation in information systems can be considered very beneficial but at the same time ethically and socially harmful. Like many other technologies, the uptake of personalisation has been rapid, with inadequate consideration given to its impact. Personalisation in e-learning systems also has potential for both harm and good but less is known...

Introduction. Digital information environments such as Websites and search engines have the potential to support serendipity: to spark and nurture positive, unexpected interactions with information and ideas with valuable outcomes. But we have few tools to measure how well they support serendipity and thus no data-driven way to guide their developm...

The session is a common unit of interaction that is used in search log analysis. By analysing sessions, it is possible to identify distinct classes of searcher behaviour that can be used to design search applications that better support groups of users based on their expected behaviours. This paper describes an online card sort experiment to invest...

Understanding information use is critical in the context of a goal-oriented information problem. We applied task analysis to examining information behaviour in the context of a work task, to understand how information was used to accomplish the task. We also propose a model for considering information use.Comprendre l’utilisation de l’information e...

Browsing is a key method by which humans obtain information, yet it is often neglected in the design of search interfaces. In this paper, we selectively examine the literature on tools developed to facilitate browsing and propose the root of a taxonomic tree of the processes that browsing facilitators need to support. Finally, we explore how a sele...

Cranfield-style evaluations standardised Information Retrieval (IR) evaluation practices, enabling the creation of programmes such as TREC, CLEF, and INEX, and long-term comparability of IR systems. However, the methodology does not translate well into the Interactive IR (IIR) domain, where the inclusion of the user into the search process and the...

The Cultural Heritage in CLEF 2013 lab comprised three tasks: multilingual ad-hoc retrieval and semantic enrichment in 13 languages (Dutch, English, German, Greek, Finnish, French, Hungarian, Italian, Norwegian, Polish, Slovenian, Spanish, and Swedish), Polish ad-hoc retrieval and the interactive task, which studied user behavior via log analysis a...

The user experience is an integral component of interactive information retrieval (IIR). However, there is a twofold problem in its measurement. Firstly, while many IIR studies have relied on a single dimension of user feedback, that of satisfaction, experience is a much more complex concept. IIR in general, and exploratory search more specifically...

Typically studies of information retrieval and interactive information retrieval concentrate on the identification of relevant items. In this study, rather than stop at finding relevant items, we considered how people use a search system in the completion of a broader work task. To conduct the study, we created 12 tasks that required multiple queri...

The interactive task in Cultural Heritage in CLEF 2013 used a standardised interactive protocol, information retrieval system and interface to observe a set of participants remotely via the web as well as in the lab access an English language collection from the Europeana Digital Library. Both user response and log data were collected from the 208...

Information Retrieval (IR) has benefited from standard evaluation practices and re-usable software components, that enable comparability between systems and experiments. However, Interactive IR (IIR) has had only very limited benefit from these developments, in part because experiments are still built using bespoke components and interfaces. In thi...

The PROMISE network of excellence organized a two-days brainstorming workshop on 30th and 31st May 2012 in Padua, Italy, to discuss and envisage future directions and perspectives for the evaluation of information access and retrieval systems in multiple languages and multiple media. This document reports on the outcomes of this event and provides...

User models of information seeking and retrieval reflect a rich culture of exploring the relationshipns among users, task, and their context. This lecture examines briefly the complex information use environment in which information retrieval systems are situated.

This talk examines interactive IR system evaluation from the holistic approach, including some of the pitfalls in existing approaches, and the issues involved in designing more effective processes and procedures.

We regularly make serendipitous discoveries in both online and offline contexts – from stumbling upon a useful website when searching for something completely different to meeting someone with mutual research or business interests in an unlikely place. However, most existing interactive systems do not provide a fertile environment for serendipity t...

- Serendipitous information retrieval is the perhaps inevitable consequence of immersion in an information-rich environment. Just how well chance encounters are supported, however, within these environments varies and one of the challenges to the development of tools and systems to facilitate serendipity is measuring how well they achieve this goal...

How do people search for information in group settings? Many assumptions are made about collaborative search from surveys and laboratory studies. Yet we have little evidence that explains how groups manage the process as our understanding is based on individual search. Using video and logfiles of groups working on complex projects, we extracted 21...

Webcasting systems were developed to provide remote access in real-time to live events. Today, these systems have an additional requirement: to accommodate the “second life” of webcasts as archival information objects. Research to date has focused on facilitating the production and storage of webcasts as well as the development of more interactive...

When does a system have the potential to enable serendipity? The objective of this research was to develop a measure, the serendipity quotient, of digital environments. By extracting indicators of serendipitous outcomes, we developed a scale and tested it with 124 participants who spent 20 minutes examining the contents of the Wikipedia using a nov...

Digital libraries have the potential to be rich interactive environments or “shariums” that support students who work in groups to complete course work. To understand how DLs might realize this potential, the processes of a single group working on a complex project over a semester were analyzed. Findings suggest that groups perform a range of tasks...

In this research, we test the hypothesis that cognitive style, (specifically Verbalizers and Visualizers) and Curiosity influence the multitasking habits of information users. The study surveyed 319 internet users about their multitasking habits with respect to a set of six information tasks and seven technology tasks, as well as the two individual...

This paper examines the validity of the User Engagement Scale (UES). Originally developed and tested in e-shopping, the scale was administered to users of a multimedia webcast system in an experimental setting. Factor analysis examined the structure and loadings of 31 items. As in previous research, a six-factor solution was found. However, the num...

Facilitating engaging user experiences is essential in the design of interactive systems. To accomplish this, it is necessary to understand the composition of this construct and how to evaluate it. Building on previous work that posited a theory of engagement and identified a core set of attributes that operationalized this construct, we constructe...

This panel will examine issues regarding the evaluation of hypertext research. The panelists will begin by contrasting four different viewpoints on the role of evaluation in hypertext research. The discussion will then consider questions relating the what evaluation methods should be used and when.

Integrating search systems with task environments to create task-specific information appliances will most likely represent the next wave of technologies. At present, search systems are for the most part isolated from the actual task environment. In this paper we have identified one task environment – that of the student writing a term paper – to p...

While serendipity is generally considered a spark for innovation and new knowledge, the triggers for serendipity appear infinite and consequently information systems' support for serendipity has been difficult to realize. Research to date has tended to focus only on supplying users with unexpected triggers for serendipity (e.g., embedded links in r...

Information about the conference is available at http://www.ht2010.org. The Proceedings are available in the ACM Digital Library http://dl.acm.org/citation.cfm?id=1810617 The general chair of Hypertext2010 was Mark Chignell and the Program Chair was Elaine Toms. HYPERTEXT 2010 was the 21st ACM Conference on Hypertext and Hypermedia. It continu...

With increasing sophistication in technology has emerged a growing interest in accessing images for personal and work purposes. In this research we investigated the use of images as data—for the information contained within the image, and as an object to illustrate. Thirty journalists and historians from academic and professional work settings were...

While many digital libraries focus on supporting defined tasks that require targeted searching, there is potential for enabling serendipitous discovery that can serve multiple purposes from aiding with the targeted search to suggesting new approaches, methods and ideas. In this research we embedded a tool in a novel interface to suggest other pages...

A digital library (DL) facilitates a search workflow process. Yet many DLs hide much of the user activity involved in the process from the user. In this research we developed an interface, wikiSearch, to support that process. This interface flattened the typical multi-page implementation into a single layer that provided multiple memory aids. The i...

Increased emphasis on user experiences with technology demonstrates that systems must be not only usable, but engaging. Engagement, defined as a quality of user experience, is a multidimensional construct characterized by aesthetic appeal, novelty, perceived challenge, feedback and control, attention, motivation, and affect. To measure engagement,...

While information processes in general have been systematically examined with respect to the individual, the same cannot be said for groups. This conceptual paper starts from existing models that represent the individual, and examines how research on group intertwines with the information world to identify an emerging framework for the study of gro...

This paper outlines a broad research agenda aimed at examining the manner in which individual differences in information seeking behavior interact with the search task to affect search outcomes. As part of this agenda, we describe specific experimentation that will assess the impact of both Need for Cognition (the tendency to elaborate upon, struct...

The analysis of search transaction logs often characterizes a search session but rarely looks at the end point. When do users stop, and what cues are present suggesting that stopping is eminent? In this preliminary analysis of the logs of 288 search sessions conducted in a laboratory setting, we identified the activity performed by participants as...

wikiSearch, is a search engine customized for the Wikipedia corpus but with design features that may be generalized to other search systems. Its features enhance basic functionality and enable more fluid interactivity while supporting both workflow in the search process and the experimental process used in lab testing.

The “task” is an important concept in Information Science, both as a theoretical and methodological tool. While many studies of information retrieval and information seeking and use take individual characteristics or system features as the starting point for their research, a growing body of work has focused on the socio-cultural perspective. This...

The purpose of this research is to examine how search differs according to selected task variables. Three types of task information goals and two types of task structures were explored. This mixed within- and between-subjects designed study had 96 participants complete three of 12 tasks in a laboratory setting using a specialized search system base...

The purpose of this article is to critically deconstruct the term engagement as it applies to peoples' experiences with technology. Through an extensive, critical multidisciplinary literature review and exploratory study of users of Web searching, online shopping, Webcasting, and gaming applications, we conceptually and operationally defined engage...

Purpose The purpose of this paper is to understand the needs of humanists with respect to information and communication technology (ICT) in order to prescribe the design of an e‐humanist's workbench. Design/methodology/approach A web‐based survey comprising over 60 questions gathered the following data from 169 humanists: profile of the humanist,...

Within the workplace setting, people need to provide sufficient access to files to allow collaboration, without inadvertently exposing sensitive files. Evidence suggests that file sharing problems exist, and decrease security and interfere with collaboration. A potential solution for managing these problems is to present the user with clear informa...

To date most of the research concerning consumer health information has focused on trust and quality of health information websites. In this research, we observed 48 consumers searching for four health-related topics (some of their own choosing) using Google. Using transaction logs, video screen capture, retrospective verbal protocols and self-repo...

In this paper, we discuss factors that influence how consumers allocate their search effort between searching within information sources (e.g., websites) and searching for relevant sources of such information (e.g., by employing general search engines). We propose that the nature of the search task as well as characteristics of the individual, infl...

Recently there has been an increased emphasis on holistic user experiences in human-computer interactions. Interface design is moving beyond usability, aiming to be aesthetically pleasing, emotionally appealing, and engaging. The term engagement is frequently mentioned in the literature as a goal of interface design, yet the construct remains abstr...

The focus of this panel is on methodologies and measures for evaluating information retrieval (IR) systems from a human-centred perspective. Current research especially with regard to search engines is challenged by “Internet time” – the need for near instantaneous results that are also reliable and valid. The session will begin with an assessment...

Faced with the rapid proliferation of digital information resources within organizations, employees need targeted search systems in order to be effective in their jobs. The goal of this study is to develop a model of information behaviour within a work-task framework that can be used to inform the design of a workplace information search system. In...

The widespread availability of broadband connections has led to an increase in the use of Internet broadcasting (webcasting). Most webcasts are archived and accessed numerous times retrospectively. In the absence of transcripts of what was said, users have difficulty searching and scanning for specific topics. This research investigates user needs...

Technical professionals spend ~25% of their time at work searching for information, and have specialized information needs that are not well-served by generic enterprise search tools. In this study, we investigated how a group of software engineers use a workplace search system. We identify patterns of search behaviour specific to this group and di...

The increased availability of broadband connections has recently led to an increase in the use of Internet broadcasting (webcasting). Most webcasts are archived and accessed numerous times retro- spectively. One of the hurdles users face when browsing and skim- ming through archives is the lack of text transcripts of the audio channel of the webcas...

Use of document genre in information retrieval systems has the potential to improve the task-appropriateness of results. However, genre classification remains a challenging problem. We describe a case study of genre classification in a software engineering workplace domain, which includes the development of a genre taxonomy and experiments in autom...

The ability to recognize documents – almost instantaneously – enhances a user's interaction with documents and makes document use more efficient. Typically in the interaction with documents people rely on signaling devices to help with the reading process as well as with the comprehension of the document. Many of these signaling devices are dictate...

As part of a large project to develop tools that aid users with the search process, we are testing ways of priming the query specification process. In this paper we test the use of questions as a technique. Using a within-subjects design 48 people interacted with a modified Google interface to solve four information problems in four domains. Half t...

Using a 48-person group in a within-subjects design, we investigated how people search differently within diverse domains: consumer health, shopping, travel and general research. Our goal was to identify the distinctive characteristics of search behavior within each domain that impact the design of the search interface.

Context influences the search process, but to date research has not definitively identified which aspects of context are the most influential for information retrieval, and thus are worthy of integration in today's retrieval systems. In this research, we isolated for examination two aspects of context: task and document genre and examined the relat...

This research assessed the effectiveness of selected interface tools in helping people respond to classic information tasks with webcasts. Rather than focus on a classic search/browse task to locate an appropriate webcast to view, our work takes place at the level of an individual webcast to assess interactivity within the contents of a single webc...

Discussion of relevance has permeated the information science literature for the past 50+ years, and yet we are no closer to resolution of the matter. In this research we developed a set of measures to operationalize the dimensions underpinning Saracevic’s manifestations of relevance. We used an existing data set collected from 48 participants who...

Webcast systems support real-time webcasting, and may also support access to the stored webcasts. Yet, research rarely examines issues concerning the interface to webcast systems, another form of multimedia system. This paper focuses specifically on how stored webcasts are re-used. Sixteen participants performed three typical information tasks usin...

The purpose of this research is to capture, understand, and model the process used by bioinformatics analysts when facing a specific scientific problem. Integrating information behavior with task analysis, we interviewed 20 bioinformatics experts about the process they follow to conduct a typical bioinformatics analysis—a functional analysis of a g...

Discussion of relevance has permeated the information science literature for the past 50+ years, and yet we are no closer to resolution of the matter. In this research we developed a set of measures to operationalize the dimensions underpinning Saracevic's manifestations of relevance. We used an existing data set collected from 48 participants who...

Context influences information seeking behaviour; however, search systems have not made much use of contextual information to date. We present research that combines information behaviour and information retrieval approaches to develop a contextual search system for a software engineering work domain. Résumé: Le contexte influence le comportement i...

In this paper, we propose a method for assessing user experience. Normally evaluation is based on usability or on the efficiency of or effectiveness of focused information search tasks. Yet all experiences with libraries (whether physical or virtual) need not be for the explicit purpose of finding, acquiring and using information. The experience an...

We introduce WiIRE, a prototype system for conducting interactive information retrieval (IIR) experiments via the Internet. We conceived WiIRE to increase validity while streamlining procedures and adding efficiencies to the conduct of IIR experiments. The system incorporates password-controlled access, online questionnaires, study instructions and...

In this study, we compare a search tool, TOPIC, with three other widely used tools that retrieve information from the Web: AltaVista, Google, and Lycos. These tools use different techniques for outputting and ranking Web sites: external link structure (TOPIC and Google) and semantic content analysis (AltaVista and Lycos). TOPIC purports to output,...