No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Usability Evaluation of Digital Libraries
Abstract
This paper distinguishes evaluation procedures in a physical library from those in a digital library, reviewing usability evaluation work within a taxonomy of system design, development and deployment. Usability evaluation of digital libraries should follow two evaluation strategies. The first a convergent methods paradigm. Evaluation data are collected throughout the system life cycle using several different methods. Where comparable interpretations result, evaluators can make confident recommendations for system revision. A second strategy assesses not only the digital library, but the evaluation methods themselves. This self-referring analysis is termed a “double-loop paradigm” and allows evaluators to identify relative efficacy of particular assessment methods for specific library situations. The paper will present evidence for adopting these two strategies, drawing upon the user evaluation effort for the Alexandria Digital Library Project.
... There is one example of this being used in a geovisualization-related project as detailed by Buttenfield (1999), who collected scenarios of use to inform the Alexandria Project. ...
... Robinson et al. (2005) and Slocum et al. (2003) used predominantly qualitative techniques as they were focussing on how people worked in geovisualization. Many studies (for example Davies 1996, Harrower 2000, Andrienko et al. 2002, Suchan 2002, Slocum et al. 2003, Robinson et al. 2005, Koua et al. 2006) use a convergent data collection methodology (using multiple techniques) as advocated by Buttenfield (1999), whereby evaluation data are collected throughout the system life cycle using several different methods, resulting in comparable interpretations. Thus, evaluators can be more confident in recommending design revisions. ...
... Standard HCI data collection techniques were used within a convergence methodology as described in section 1.3, allowing comparable interpretation and enabling evaluators to be more confident in identifying patterns of use and activity and in recommending design revisions (Buttenfield, 1999). No individual HCI techniques were adapted, the overall methodology as seen in Focus group (4 students) Individual interviews (2 students) Questionnaires (All students) To record the students' opinions and attitudes towards the software and the exercise. ...
... UCD describes an early and active focus on the needs of the user when conceptualizing and implementing an interface [2], with an emphasis on iterative refinement to the ease-of-use and usefulness of the interface [3,4]. UCD increasingly has been recommended for interactive maps, e.g., [5,[6][7][8][9], and has been leveraged within GIScience for the design and evaluation of digital geospatial libraries [10], desktop geovisualization tools [11][12][13], mobile mapping applications [14], participatory mapping tools [15,16], spatial decision support tools [17], virtual environments [18,19], and web mapping applications [20][21][22][23], among others. However, preliminary evidence suggests that UCD may not be common in practice, despite the desire of interactive map users to be more involved in the conceptualization, evaluation, and refinement of their interactive mapping systems [24]. ...
... Several scholars organize the available array of interface evaluation methods according to the recommended stage in the UCD process during which the method should be applied. For example, Buttenfield [10] classifies interface evaluation methods into three categories according to the stage in the overall process: (1) design (e.g., participant observation, needs assessment interviews); (2) development (e.g., cognitive walkthroughs, conformity assessment); and (3) deployment (e.g., automated evaluation, entry/exit surveys). Several popular web resources regarding usability engineering use similar, threepart classifications of interface evaluation methods according to stage (Table 1). ...
... To circumvent this issue, Nielson [4] recommends a discount approach to user-based interface evaluation, recruiting only a small number of participants (3-5 target users) for each evaluation, with reliability maintained by triangulating insights across multiple user→utility→usability loops. Buttenfield [10] describes the administration of multiple, discount empirical evaluations during UCD as the convergent methods paradigm. Table 2 enumerates interface evaluation methods commonly drawn from each of our three evaluator-based categories. ...
In this paper, we address the topic of user-centered design (UCD) for cartography, GIScience, and visual analytics. Interactive maps are ubiquitous in modern society, yet they often fail to “work” as they could or should. UCD describes the process of ensuring interface success—map-based or otherwise—by gathering input and feedback from target users throughout the design and development of the interface. We contribute to the expanding literature on UCD for interactive maps in two ways. First, we synthesize core concepts on
UCD from cartography and related fields, as well as offer new ideas, in order to organize existing frameworks and recommendations regarding the UCD of interactive maps. Second, we report on a case study UCD process for GeoVISTA CrimeViz, an interactive and web-based mapping application supporting visual analytics of criminal activity in space and time. The GeoVISTA CrimeViz concept and interface were improved iteratively by working
through a series of user→utility→usability loops in which target users provided input and feedback on needs and designs (user), prompting revisions to the conceptualization and functional requirements of the interface (utility), and ultimately leading to new mockups and prototypes of the interface (usability) for additional evaluation by target users (user… and so on). Together, the background review and case study offer guidance for applying UCD to
interactive mapping projects, and demonstrate the benefit of including target users throughout design and development.
... We discuss possible reasons for incorrect assumptions in users' mental models later. Buttenfield (1999) cites Lancaster (1995, who argues that while the digital library implies some fundamental conceptual differences [from the physical one], most obviously document access instead of document delivery, "user objectives and evaluation criteria will not change in substance even where contents of the digital library may differ most profoundly from those of a traditional library (e.g. non-static items versus static items, items with fuzzy boundaries versus items with precise boundaries)." ...
... However some similarities between traditional and digital libraries do certainly exist (and are arguably less radical that the translation of entities proposed by Nürnberg et al., 1995). According to Buttenfield (1999), "user needs for information will modify as they gather information. Patrons expect to focus on topical interests, not in struggling to learn the library organisation (Kemeny, 1965). ...
... The intention is to shorten the learning curve." Buttenfield (1999) argues that this drives the logic underlying traditional library organisation (books in the main stacks, current periodicals in a special reading room, maps and large or heavy artefacts in the basement etc.) (Buttenfield, 1999). This may allow for the transfer of mental models from one specific library system to another and hence can be deemed as valuable in aiding users to form richer mental models of groups of libraries. ...
Project report submitted in part fulfilment of the requirements for the degree of Master of Science (Human-Computer Interaction with Ergonomics) in the Faculty of Life Sciences, University College London, 2004. NOTE BY THE UNIVERSITY This project report is submitted as an examination paper. No responsibility can be held by London University for the accuracy or completeness of the material therein.
... Her research efforts in the ADL extended the role of cartographers from traditional map designers to user interface designers, focusing on web-based user interfaces and user experiences (UI/UX). This paper will highlight the key concepts and design principles (including user center design, usability testing, and user interface evaluation) in the ADL project (Buttenfield, 1999) and reinvestigate these pioneering research topics today in the new 3D/4D environments using XR technologies (such as VR, AR, and MR). ...
... These lessons can help cartographers to design more effective GIS application in a 3D/4D Metaverse platform using XR-enabled technologies (VR, AR, and MR). In the ADL projects, cartographers were collaborating with over 20 experts from multiple disciplines (including computer scientists, librarians, cognitive science researchers, GIScientists, cartographers, management information systems (MIS) experts, environmental scientists, educators, electronic engineers, user interface designers, and art studio members) (Buttenfield, 1999). Transdisciplinary collaborations are essential today for building user-friendly and effective 3D GIS applications and multi-user mapping services. ...
... Demystifying the target users can be done through stage one of UCD, needs assessment. This stage reflects a formative evaluation that is conducted early in the design process and aimed to solicit feedback that can help improve the interface compared to a summative evaluation that takes place after completion of an interface to compare the new release with an old design or similar products (Buttenfield, 1999). ...
... Focus groups are not meant to be used for confirmatory research; rather, focus groups should be used in exploratory research to assess how and why people respond to stimuli (Harrower et al., 2000). Thus, focus groups can be characterized as a formative evaluation that aims to solicit feedback on earlier iterations of a particular interface (Buttenfield, 1999). Focus groups can be used both early in the UCD process as a way to characterize target users or later to evaluate a user interface in a less structured manner (Roth et al., 2015). ...
Interactive maps can serve as powerful environmental decision-support tools. However, designing an interactive map that meets the needs of diverse constituencies is a challenge. In this article, we evaluate and characterize user needs for an interactive map and spatial decision-support tool called Beescape. Beescape is designed to visualize resources and environmental risks to bees and other pollinators (such as availability of nutritional resources from flowering plants and exposure to pesticides) in order to help users make informed decisions about managing bee populations and associated landscapes. We conducted a needs assessment workshop with twenty stakeholders from four user groups including beekeepers, growers, conservationists, and pollinator scientists to elicit their knowledge to guide future Beescape development. The results of the workshop identify current analytical gaps with the existing Beescape prototype, including the need for predictive and historical tools, more actionable data layers, finer-grain spatial data, and better explanations on what data represent and how they were created. Our findings on user's analytical, informational, and interface needs can be utilized to guide the future development of spatial decision support tools like Beescape, and our methodological approach may apply to other environmental informatics tools where it is important to design for multiple constituent user groups.
... Finally, an interface may be evaluated after deployment while it is being actively used, described here as post-deployment evaluation, which can lead to interface improvements even after it has formally launched (Buttenfield 1999). This feedback can be retrieved in multiple ways, including interviews and focus groups with users of the interface, continuous userperformance data logging, online or telephone feedback, and wikis (Shneiderman and Plaisant 2010). ...
... There are numerous methods for evaluating interface success, each with trade-offs according to the stage in the development process, the goals of the evaluation, the type of feedback the evaluation provides, and the cost of conducting the method (Buttenfield 1999;Robinson et al. 2005). argue that the best way to describe evaluation methods is to look at the person performing the interface evaluation, and that different methods are better suited to different kinds of interface evaluators. ...
Advances in personal computing and information technologies have fundamentally transformed how maps are produced and consumed, as many maps today are highly interactive and delivered online or through mobile devices. Accordingly, we need to consider interaction as a fundamental complement to representation in cartography and visualization. UI (user interface) / UX (user experience) describes a set of concepts, guidelines, and workflows for critically thinking about the design and use of an interactive product, map-based or otherwise. This entry introduces core concepts from UI/UX design important to cartography and visualization, focusing on issues related to visual design. First, a fundamental distinction is made between the use of an interface as a tool and the broader experience of an interaction, a distinction that separates UI design and UX design. Norman’s stages of interaction framework then is summarized as a guiding model for understanding the user experience with interactive maps, noting how different UX design solutions can be applied to breakdowns at different stages of the interaction. Finally, three dimensions of UI design are described: the fundamental interaction operators that form the basic building blocks of an interface, interface styles that implement these operator primitives, and recommendations for visual design of an interface.
... To address these research questions, we designed and executed a repeatable process following the discount, convergent approach recommended in the fields of usability engineering and user-centered design (Buttenfield 1999). ...
... We designed and executed a three-stage process in order to characterize and push our way into the current landscape of open source web mapping technologies. Design of the process followed the convergent methods paradigm, which prescribes administration of multiple, often qualitative methods (Buttenfield 1999). Each study then is conducted in a discount manner (e.g., leveraging secondary sources, recruiting only a small number of participants) to ascertain input and feedback quickly (Nielsen 1993). ...
... To address these research questions, we designed and executed a repeatable process following the discount, convergent approach recommended in the fields of usability engineering and user-centered design (Buttenfield 1999). ...
... We designed and executed a three-stage process in order to characterize and push our way into the current landscape of open source web mapping technologies. Design of the process followed the convergent methods paradigm, which prescribes administration of multiple, often qualitative methods (Buttenfield 1999). Each study then is conducted in a discount manner (e.g., leveraging secondary sources, recruiting only a small number of participants) to ascertain input and feedback quickly (Nielsen 1993). ...
The current pace of technological innovation in web mapping offers new opportunities and creates new challenges for web cartographers. The continual development of new technological solutions produces a fundamental tension: the more flexible and expansive web mapping options become, the more difficult it is to maintain fluency in the teaching and application of these technologies. We addressed this tension by completing a three-stage, empirical process for understanding how best to learn and implement contemporary web mapping technologies. To narrow our investigation, we focused upon education
at the university level, rather than a professional production environment, and upon open source client-side web mapping technologies, rather than complementary server-side or cloud-based technologies. The process comprised three studies: (1) a competitive analysis study of contemporary web mapping technologies, (2) a needs assessment survey of web map designers/developers regarding past experiences with these technologies, and (3) a diary study charting the implementation of a subset of potentially viable technologies, as identified through the first two studies. The process successfully achieved the practical goal of identifying a candidate set of web mapping technologies for teaching web mapping, and also revealed broader insights into web map design and education generally as well as ways to cope with evolving web
mapping technologies.
... Several authors agree (Aula & Käki, 2005;Buttenfield, 1999;Hargittai, 2004;Hearst, 2009), that one of the fundamental principles for building search interfaces focus on the simplicity of them. For that reason, it is very important to analyze the interface from the point of view of the users, which is the main aim of HCI, in order to maximize user satisfaction. ...
... For that reason, it is very important to analyze the interface from the point of view of the users, which is the main aim of HCI, in order to maximize user satisfaction. Therefore, usability strategies within the HCI field are an important factor in the development and evolution of the interfaces found in both libraries and digital repositories (Buttenfield, 1999;Fox et al., 1999;White & Roth, 2009). Other studies (G. ...
In this paper, it is presented a study to evaluate and compare the effectiveness of eight different interfaces based on visualization techniques to carry out searching digital resources according to knowledge area and knowledge representation scheme. The navigation structure is based on hierarchical taxonomic representation through the use of Art and Architecture Thesaurus (AAT). It was adopted principles of Human Computer Interaction (HCI) for the development of search interfaces through information visualization techniques. Finally, it will be presented preliminary results in order to evaluate the most representative aspects to carry out the development of visual search interfaces of learning objects in digital repositories, according to principles of visualization techniques and hierarchic taxonomic classification.
... Goal 2 address the question: "Can accessible flood maps foster collaboration between B/ LV and sighted community members?" This study is a formative assessment (Buttenfield 1999), meaning that it is one step in a series of design iterations, as opposed to a summative assessment that compares a new design against an existing one. ...
Conventional visual maps present significant accessibility challenges for blind or low vision users, leaving them with few or no options for interpreting spatial data. This need not be the case: tactile maps, designed to be read through touch, have been published for more than a century. But they have most often been categorized as a navigation tool, or mere “tactile graphics” (i.e., not as expressly spatial documents). Tactile maps that allow their users to explore and synthesize thematic spatial data are rare, as are studies evaluating them. As our world continues to face existential threats that are spatial in nature—pandemics, supply chain disruptions, floods, etc.—maps will continue to provide critical information in ways that other media are unable to match. In the absence of accessible thematic maps, blind people will not only be left out of the loop, but their capacity for contributing valuable input will be severely diminished. In response, I describe here a study that evaluates the potential of thematic tactile maps for providing blind users an accessible means of analyzing spatial data when working in collaboration with sighted partners. Findings indicate that while the maps did not prove to be useful tools on their own, they did facilitate collaboration between blind or low vision participants and sighted participants. This suggests that, with some refinements, similar maps could be feasibly distributed as a means for people with visual disabilities to meaningfully participate in an otherwise inaccessible process that requires the synthesis of thematic spatial information.
... The study we present here corresponds to stage four of the UCD process, interaction and usability studies. To complete this stage of work, we conducted a formative evaluation (Buttenfield, 1999) of Beescape in the form of an online task analysis and survey. Usability and utility were the primary lenses by which we used to formulate our tasks and survey questions. ...
Pollinators, particularly managed honey bees, are crucial for global food systems. However, declines in populations of both wild and managed pollinators have been reported across the world. In the United States, approximately 30% of managed honey bee colonies die each year. The factors underlying these losses are well understood and include reductions in the abundance and diversity of flowering plants that pollinators depend on for food, increased insecticide use, and reduced nesting habitat for wild bees. However, translating pollinator research findings into actionable knowledge for beekeepers presents a sizable spatial decision support challenge. In this work we evaluate the utility and usability of a prototype system called Beescape which intends to support environmental decision-making for beekeepers and other stakeholders. Beescape includes tools for exploring and visualizing maps that link to modeled environmental factors that impact managed bees, including honey bees, and wild bee health. Thirty beekeepers were recruited to take part in an online user study that included task analysis and survey components to elicit user input on areas of improvement for future Beescape development. The results of our evaluation of usability and utility metrics for Beescape highlighted the need for spatially-specific foraging information, better descriptions of modeled habitat quality measures, and actionable guidance to help them manage their hives. The results of our study highlight the challenges and opportunities associated with providing decision support systems that attempt to translate emerging environmental science for audiences that may be motivated by a common goal to improve honey bee survival, but who have a diverse range of technical backgrounds, applied practices, and reasons for their interest.
... It has been argued that simplicity is a fundamental principle of building search interfaces (Aula and Käki 2005;Buttenfield 1999). This is why interface analy sis from the user perspective is one of the most prevalent methods of CHDL evaluation identified in this study as well as one of the primary focuses of the human-computer interaction field. ...
Digital library evaluation has become increasingly important in information science, yet there has been minimal evaluative work focusing on digital cultural heritage. We report on a comprehensive review of methodologies and frameworks used in the evaluation of cultural heritage digital libraries and archives. Empirical studies are examined using Tefko Saracevic's digital library evaluation framework to identify models, frameworks, and methodologies in the literature and to categorize these past evaluative approaches. Through the classification and critique of evaluative types and trends, we aim to develop a set of recommendations for the future evaluation of cultural heritage digital libraries and archives.
... Studies examining the use of search engines in digital libraries and archives have analyzed patterns of use, developed models for the optimal retrieval of desired results, and described the intricacies of user experience (Buttenfield, 1999;Huvila, 2008;Tsakonas et al., 2004). Users' ability to formulate a query that will enable the retrieval of pertinent information is, of course, important. ...
This article examines the construction of a digital collection. Using a theoretical framework adapted from digital history and historiography, it will investigate the implications of archival digitization. Through an empirical study of the National Library of Israel’s digital depository of ephemera entitled ‘Time Travel’, the article demonstrates how the selection of archival records for digital preservation, the design of the search interface, and the crowdsourcing of metadata collection are all directing archive users toward certain narratives about Israeli history and away from others. Drawing on interviews with professionals, analysis of reports, and investigations of user experience, I will unearth the political, religious, and cultural tensions that lie beneath the surface of ‘Time Travel’. This research demonstrates that digitization of archival documents is not just a technical process but a cultural, social, and political one as well.
... They are progressively worried about the structure of a user's point of view into the design cycle than with final assessment. Buttenfield (1999) did examination and proposed two assessment methodologies for ease of use investigations of digital libraries: the convergent method paradigm that applies the system life cycle into the assessment procedure and the double-loop paradigm that empowers evaluators to recognize the estimation of a specific assessment technique under various circumstances. Despite the fact that convenience is generally talked about, he also talked about it is important to recognize the uniqueness of ease of use traits for the evaluation of digital libraries. ...
This study has been conducted for literature review on Impact of ICT on learning activities of users by Academic library services. The study focuses on the technology utilized in Academic libraries and impact on discovering how ICT keeps on advancing, and as it does it is turning out to be increasingly more incorporated with students and researcher learning. Brilliant machines and voice-controlled aides are only two instances of how technology is developing to make students' careers simpler. This study explains how technology changes the manner in which library services work and it additionally influences the way learning exercises of users.
... Eye tracking was the principal method used, as it allowed us to investigate (in unobtrusive manner) how untrained individuals get started with a CMV interface. However, as converging methods are important for assessing complex geospatial interfaces (Buttenfield, 1999), we also used interaction logs and the talking aloud technique. This combination of methods provided us with a comprehensive insight into how first-time users explore a CMV geovisualization tool. ...
Maps are frequently combined with data displays in the form of coordinated and multiple views (CMV). Although CMV are valuable geovisualization tools, novice users may find them complex and thus require explanation. However, no tutorial guidelines have been developed that indicate what is helpful in understanding CMV geovisualization tools. We therefore conducted a study on the learnability of a CMV tool, informed with eye-tracking data, talk-aloud and interaction logs. We have investigated how untrained users work with a CMV geovisualization tool. The study revealed that: (1) despite their initial confusion, users found the tested tool pleasant to play with while getting to grips with how dynamic brushing works, (2) when examining the tool's interface, participants mainly looked freely at explanatory elements, such as labels and the legend, but they explored interactive techniques only to a limited degree. We conclude with tips about tutorial design and layout design for CMV tools.
... While common reasons for not following a user-centered dev cycle include perceived constraints on time, money, and expertise, a user-centered process instead saves resources by facilitating faster agreement about design decisions early in the project and avoiding unplanned revisions late in the project. To make efficient use of project resources, "discount" methods are employed to rapidly collect feedback from 3-5 target users at multiple stages in design and development (Nielsen 1994), with insights converging towards the final, successful design (Buttenfield 1999). how do user-centered design studies contribute to cartography? ...
I ask in this essay: How do user-centered design studies contribute to cartography?
Scholars in related fields increasingly recognize the intellectual value of employing user-centered processes to improve a single product and identify new design considerations for future products. To this end, I propose an analytical framework for organizing the contributions of user-centered design studies that includes eight opportunities for advancing cartography: (1) domain gap analyses, (2) adapted or novel user-centered methods, (3) streamlined user-centered design processes, (4) transferable design insights, (5) comprehensive user-centered design case studies, (6) novel or unique maps and visualizations, (7) summative controlled experiments, and (8) new insights into pressing geographic problems. I apply this framework against my own collaborative work in a retrospective analysis of three UCD case studies: the GeoVISTA CrimeViz visual analytics tool, the NOAA Lake Level Viewer climate change visualization, and the UW Cart
Lab Global Madison mobile map.
... Interlibrary loan requests have also significantly decreased since the introduction of online journals (F(2,30) = 4.46, P < 0.02). Buttenfield (1999) established a correlation between in-house use, circulation, and citation by faculty, which suggests that the gathering of many types of data is impractical and that one method may be used with the confidence that it correlates with other types of uses. Electronic information resources are increasingly in use for cores at all levels of higher education (Macdonald, Heap & Mason, 2001). ...
The study aims at investigating the awareness and usage of electronic information resources among postgraduate students of library and information science in Southern Nigeria. The descriptive survey design was adopted for the study. The census sampling technique was adopted for this study. Thus, the entire population of three hundred and seventy-five (375) postgraduate students of library and information science in Southern Nigeria were used as the sample for this study. The questionnaire tagged: Awareness and Usage of Electronic Information Resources by Postgraduate Students of Library and Information Science Questionnaire (AUEIRPSLISQ) was used as instrument for data collection. Four research questions were answered and two null hypotheses were tested at 0.05 level of significance. The simple percent statistical tool was used to answer the research questions and Pearson Product Moment Correlation Coefficient (PPMCC) for testing the hypotheses. The results obtained revealed that postgraduate students of library and information science are quite aware and highly use electronic information resources. The study also reported that postgraduate LIS students are skilled in the use of electronic information resources. Based on the findings the study concluded that electronic information resources are essential tools for empowering postgraduate students of library and information science in Southern Nigeria.
... Several authors (Aula and Käki, 2005;Buttenfield, 1999;Hearst, 2009) argue that simplicity should be one of the fundamental principles of building search interfaces. For this reason, it is important to analyse an interface from a user's point of view to maximise user satisfaction, which is the main aim of the field of HCI. ...
Purpose
This paper aims to present an overview of the challenges encountered in integrating visual search interfaces into digital libraries and repositories. These challenges come in various forms, including information visualisation, the use of knowledge organisation systems and metadata quality. The main purpose of this study is the identification of criteria for the evaluation and integration of visual search interfaces, proposing guidelines and recommendations to improve information retrieval tasks with emphasis on the education-al context.
Design/methodology/approach
The information included in this study was collected based on a systematic literature review approach. The main information sources were explored in several digital libraries, including Science Direct, Scopus, ACM and IEEE, and include journal articles, conference proceedings, books, European project reports and deliverables and PhD theses published in an electronic format. A total of 142 studies comprised the review.
Findings
There are several issues that authors did not fully discuss in this literature review study; more specific, aspects associated with access of digital resources in digital libraries and repositories based on human computer interaction, i.e. usability and learnability of user interfaces; design of a suitable navigation method of search based on simple knowledge organisation schemes; and the use of usefulness of visual search interfaces to locate relevant resources.
Research limitations/implications
The main steps for carrying out a systematic review are drawn from health care; this methodology is not commonly used in fields such as digital libraries and repositories. The authors aimed to apply the fundamentals of the systematic literature review methodology considering the context of this study. Additionally, there are several aspects of accessibility that were not considered in the study, such as accessibility to content for disabled people as defined by ISO/IEC 40500:2012.
Originality/value
No other systematic literature reviews have been conducted in this field. The research presents an in-depth analysis of the criteria associated with searching and navigation methods based on the systematic literature review approach. The analysis is relevant for researchers in the field of digital library and repository creation in that it may direct them to considerations in designing and implementing visual search interfaces based on the use of information visualisation.
... Since people tend to look at the things they are thinking about (Nielsen and Pernice 2010), we asked the participants to solve five tasks, and recorded their eye movements during task execution. As converging methods are essential for assessing complex geospatial interfaces (Buttenfield 1999, Robinson et al. 2005, we combined eye-tracking with usability metrics (time and accuracy of answers). The study was conducted in an area used for controlled audio and visual perception experiments: the Speech, Cognition and Language Laboratory (SCaLa) at the Department of Psychology of the Norwegian University of Science and Technology (NTNU). ...
Geographic visualization tools with coordinated and multiple views (CMV) typically provide sets of visualization methods. Such configuration gives users the possibility of investigating data in various visual contexts; however, it can be confusing due to the multiplicity of visual components and interactive functions. We addressed this challenge and conducted an empirical study on how a CMV tool, consisting of a map, a parallel coordinate plot (PCP), and a table, is used to acquire information. We combined a task-based approach with eye-tracking and usability metrics since these methods provide comprehensive insights into users’ behaviour. Our empirical study revealed that the freedom to choose visualization components is appreciated by users. The individuals worked with all the available visualization methods and they often used more than one visualization method when executing tasks. Different views were used in different ways by various individuals, but in a similarly effective way. Even PCP, which is claimed to be problematic, was found to be a handy way of exploring data when accompanied by interactive functions.
... To understand the extent to which we met these goals, participants provided written responses to task prompts and survey questions as described in the next several sections. Our study is therefore a formative evaluation, intended to inspire the next round of design refinements for our approach and tools, rather than a summative evaluation in which we might have collected interaction logs or other quantitative data in order to evaluate raw system performance in terms of task time and errors (Buttenfield 1999). ...
It remains difficult to develop a clear understanding of geo-located events and their relationships to one another, particularly when it comes to identifying patterns of events in less-structured textual sources, such as news feeds and social media streams. Here we present a geovisualization tool that can leverage computational methods, such as T-pattern analysis, for extracting patterns of interest from event data streams. Our system, STempo, includes coordinated-view geovisualization components designed to support visual exploration and analysis of event data, and patterns extracted from those data, in terms of time, geography, and content. Through a user evaluation, we explore the usability and utility of STempo for understanding patterns of recent political, social, economic, and military events in Syria.
... One of the first ones highlighting the necessity of mixed-research design in the proposition of general experiment was Buttenfield (1999), who used several complementary methods of quantitative but also qualitative nature in her research. As already mentioned, it is necessary to emphasize the structure and design of a test (Olson, 2009). ...
The first part of this publication deals with theoretical aspects of the evaluation of cartographic products. Existing evaluation approaches of cartographic products are described, from strictly subjective evaluation methods to objective methods focusing on usability of cartographic visualizations. The obvious emphasis is put on psychological aspects, which can have a significant effect on the communication of information between the map and the user. From this point of view, there has been a detailed discussion of the phenomenon of cognitive style, which brings the possibility of studying individual differences among users of cartographic products. Options for testing cognitive styles among users, and of course, also, in connection with the activities associated with work on the map, are then presented in this sense. The second part of this publication is focused on the practical use of newly developed interactive testing software Hypothesis, which has been used in experimental research in cartography. Simple examples present the functionality of this tool, which enables the implementation of objective and subjective evaluation methods and testing the user’s performance on the map, according to the requirements of the specific research project.
... Numerous studies in geovisualization have used domain expert knowledge in the process of designing tools to facilitate exploration and discovery with geographic data (Buttenfield 1999;Harrower et al. 2000;Edsall 2003;Acevedo et al. 2008). Responding to a call from MacEachren and Kraak (2001) to, "…develop a comprehensive user-centered design approach to geovisualization usability," Robinson et al. (2005) combined a range of usability techniques to design geovisualization tools, developing a process that places Work Domain Analysis at the forefront of the process, but also placing user participation at every stage. ...
Animated maps provide an intuitive method for representing univariate time-series data,
but often fail in presenting additional relevant information saliently, making recognition of
certain patterns difficult. Using a second visual variable in animations to represent the magnitude
of change between time states has been suggested as an effective method for enabling users to
more easily recognize patterns of change in a geographic time-series. This work seeks to answer
the question: Does explicitly representing geographic change in animated maps enable users to
answer questions about patterns of change easily? To address this research question, bivariate
symbols (with both the value of the data and the magnitude of change between time frames
represented) were created and tested. Selective attention theory (SAT) was used in selecting
bivariate symbol types (separable and integral). Domain analysis with experts from the Avian
Knowledge Network (AKN) was performed to determine appropriate map reading tasks for use in
task-based experiments using AKN data. Combined with existing task typologies, material from
the domain analysis helped form a new task typology of movement patterns found in aggregated
spatiotemporal point data. Formal task-based experiments followed, where participants were
placed into one of five experiment groups (each using a different symbol) and asked to perform
the same series of statement agreement and certainty ratings while studying map animations.
Results show that aside from questions explicitly about change, univariate non-change
symbolization may be most appropriate. Future studies should focus on testing different data
relationships (independent, interdependent, or unrelated) with symbol variations that may have
different attention behaviors as predicted by SAT. The results presented here improve the
understanding of whether explicit change symbolization helps elucidate geographic time-series
patterns or hinders the overall effectiveness of map animation.
... Some multi-faceted evaluations are reported in the literature (see, for instance, Hill et al. 2000, on the Alexandria Digital Library). 'Convergent methods' evaluation is convincingly advocated by Buttenfield (1999), who draws together ethnography, cognitive walk-throughs, videorecordings, online surveys, focus groups and TLA. As the usability testing approach gains currency in LIS, so the synthesis of various techniques is likely to increase, given that usability is commonly defined by instruments with extensive sets of measures, demanding a sophisticated combination of methods. ...
Libraries and other information agencies have highly developed systems, skills, and techniques for delivering information to users. There is, nevertheless, a need to improve the delivery systems and discusses the means by which web-based systems in particular can be evaluated. The authors review systems evaluation in recent LIS literature and identify some of the common measures and methodologies employed.
... Knowledge of both conditions can be acquired through early and active input from targeted end users, a process often referred to as user-centered design [200]. User-centered design is becoming increasingly popular in cartography [21,27,85,86,105,106,123,228,245,262,273]. Rather than repeat these efforts to transition the tenets of usercentered design to cartography, the following review of the who? question identifies three user characteristics that impact the quality of cartographic interactions and thus need to be considered during user-centered design: ability, expertise, and motivation. ...
This article provides a review of the current state of science regarding cartographic
interaction, a complement to the traditional focus within cartography on cartographic representation.
Cartographic interaction is defined as the dialog between a human and map,
mediated through a computing device, and is essential to the research into interactive cartography,
geovisualization, and geovisual analytics. The review is structured around six
fundamental questions facing a science of cartographic interaction: (1) what is cartographic
interaction (e.g., digital versus analog interactions, interaction versus interfaces, stages of
interaction, interactive maps versus mapping systems versus map mash-ups); (2) why provide
cartographic interaction (e.g., visual thinking, geographic insight, the stages of science,
the cartographic problematic); (3) when should cartographic interaction be provided
(e.g., static versus interactive maps, interface complexity, the productivity paradox, flexibility
versus constraint, work versus enabling interactions); (4) who should be provided with
cartographic interaction (e.g., user-centered design, user ability, expertise, and motivation,
adaptive cartography and geocollaboration); (5) where should cartographic interaction be
provided (e.g., input capabilities, bandwidth and processing power, display capabilities,
mobile mapping and location-based services); and (6) how should cartographic interaction
be provided (e.g., interaction primitives, objective-based versus operator-based versus
operand-based taxonomies, interface styles, interface design)? The article concludes with
a summary of research questions facing cartographic interaction and offers an outlook for
cartography as a field of study moving forward.
... Despite this fact, as noted by Olson (2009), combinations of quantitative and qualitative designs are often neglected in cartography. One of the first authors who stressed the importance of mixed research designs in experiments was Buttenfield (1999), who used several supplementary methods in her study, representing both quantitative and qualitative approaches. ...
The contribution deals with methodological issues connected with the evaluation of cartographic materials. A new research design for map evaluation comes from an extensive cooperation between cartographers and psychologists. Contrary to strictly quantitative approaches, we propose a mixed research design, combining the quantitative and qualitative methods, because also the qualitative exploration of the corresponding cognitive strategies is in some cases necessary for an in-depth interpretation of the results. Proposed research design is demonstrated by a sample experiment concerning comparison and evaluation of suitability of two different cartographic approaches to avalanche hazard visualization. The quantitative analysis enables to detect potential differences in the compared visualization methods in the sense of their effectiveness and efficiency, and the qualitative analysis of the eye-movement data might improve our understanding of the ways different cognitive strategies are applied. The experiment was carried on the newly developed interactive assessment tool MUTEP, which substantially increases the reliability of experiments in cognitive cartography. MUTEP enables easy data collection and effective evaluation of the results which could be subsequently analyzed.
... A cartographic "front end" helps the user of a digital library search for images, maps, or other environmental data and metadata. The best known example is the Alexandria Digital Library (Buttenfield 1999). Other government agencies, including the U.S. Geological Survey, also provide distributed data, though not necessarily interactively. ...
My intent in this paper is to answer two questions: what were the principal events in the development of distributed mapping, and how should a narrative of its development be written? Distributed mapping is a mode of cartography arising from the convergence of the World Wide Web, GIS, and digital cartography. It marks a significant break with traditional cartography because (1) the set of rules that shape the map archive are being fundamentally altered; (2) the distributivity of spatial data, their analysis and visualization are at unprecedented levels; and (3) new forms of interactivity are emerging. After discussing some theoretical issues in the history of cartography, I locate the multiple origins of distributed mapping in the work on animated mapping during the quantitative revolution in geography and the availability of computing power from the 1960s through the 1980s. The technology is a series of non-deterministic negotiations with resistance leading to delays in implementation, back-tracking, and multiple avenues of exploration. The popularization of the World Wide Web during the latter part of the 1990s brought commercial attention to distributed mapping, not as cartography, but as support service for travel sales channels. Commercialization will detach distributed mapping from academic geography as it did with GIS before it. In conclusion, I outline the forseeable research issues for distributed mapping.
... A cartographic "front end" helps the user of a digital library search for images, maps, or other environmental data and metadata. The best known example is the Alexandria Digital Library (Buttenfield 1999). Other government agencies, including the U.S. Geological Survey, also provide distributed data, though not necessarily interactively. ...
... DLs have their own unique characteristics and features compared to traditional library services. Also, as an Libraries Initiatives I and II, laid groundwork in evaluation research by producing DL prototypes and frameworks (Borgman et al. 2000;Buttenfield 1999;Hill et al. 2000;Van House et al. 1996). In particular, Hill et al. (1997) identified several criteria for digital library evaluation, such as ease of use, overall appeal, usefulness and overall performance. ...
Criteria for evaluating digital libraries have been suggested in prior studies, but limited research has been done to understand users' perceptions of evaluation criteria. This study investigates users' opinions of the importance of digital library evaluation criteria. Thirty user participants, including 10 faculty members and 20 students, were recruited from five universities across the United States. They were asked to rate the importance of evaluation criteria in eight dimensions (e.g. collections, information organization, context, etc.). The results demonstrate that users care about evaluation criteria related to the use and quality of collection and services rather than the operation of digital libraries. The findings of this study are relevant to the development of user-centered digital libraries and associated evaluation frameworks through the incorporation of unique users' needs and preferences.
... Extant literature has revealed factors which promote or hinder the adoption and usage of ICT especially in digital libraries including: benefits/ usefulness: (Bar-Ilan, Peritz, & Wolman, 2003;Baruchson-Arbib & Shor, 2002;Entlich et al., 1996;Harless & Allen, 1999;Marchonini, 2000;Theng et al., 2007;, awareness (Bar-Ilan, Peritz, & Wolman, 2003;Bishop, 2002;Harless & Allen, 1999;Nicholson, 2004), relevance (Kwak et al., 2002;Nicholson, 2004;Vaidyanathan et al., 2005;Nov & Ye, 2008), among others, and ease of use (Buttenfield, 1999;Lagier, 2002;Nov & Ye, 2008) and many others. ...
Universities are investing heavily in electronic resources. As a way of embracing new developments, the University of Ilorin, Nigeria, has spent millions of dollars building a usable e-library. However, research indicates that potential users may still not use e-libraries. This study examines user acceptance of e-library from the perspective of technology acceptance mode TAM. E-library system characteristics, organisational context, and individual characteristics are identified as variables that determine acceptance. Data was collected through self-designed questionnaire from 1,500 undergraduate users of the e-library. The findings revealed that the acceptance constructs, ease of use, perceived usefulness, actual use, satisfaction, relevance, awareness, computer/internet self-efficacy, and social influence, significantly correlate with e-library acceptance. The study suggests that all eight factors jointly pulled 69% prediction of the users' acceptance of e-library. The study recommends that e-library users at the university increase their computer and internet self-efficacy, which significantly enhances their use of the e-library system. The university can assist in this matter by organising computer training for the students.
... Many of the survey questions also prompted users to provide qualitative feedback to elaborate upon their opinions and suggest specific new features or bugs to fix. This user study was aimed at formative assessment (Buttenfield 1999) of the Symbol Store prototype in order to establish which steps to take going forward to add/remove key features and to develop a baseline understanding of the tool's usability. ...
Maps are a primary means for supporting information sharing and collaboration in emergency
management and crisis situations. While a variety of formalized map symbol standards for emergency
contexts exist, they have not been widely adopted by mapmakers. Informal symbol conventions are
commonly used within emergency management stakeholder groups, but until now there has not been a
flexible mechanism for discovering, sharing, and previewing these symbol sets among mapmakers. In
this paper we describe the design and development of the Symbol Store, a visually-enabled, web-based
interactive tool intended to help mapmakers share point symbols. The Symbol Store allows users to
browse symbols by keyword, category tags, and contributors. It also allows for symbols to be previewed
on realistic maps prior to download. An initial prototype of the Symbol Store was evaluated by flood
mapping experts from the State of California, and the results of this user study led to multiple
refinements now implemented in the public version of Symbol Store located at www.symbolstore.org.
... In addition, this research examines the issues of user lostness and navigation disorientation. The user lostness issue has been reported by several studies, including Blandford, Stelmaszewska, and Bryan-Kinns (2001), Buttenfield (1999), Gullikson et al. (1999), Kengeri et al. (1999), andSpool et al. (1999) as well as by Theng, Mohd-Nasir, and Thimbleby (2000a). Indeed, navigation disorientation is among the biggest frustrations for Web users (Brinck, Gergle, and Wood 2002). ...
This paper reviews how usability has been defined in the context of the digital library, what methods have been applied and their applicability, and proposes an evaluation model and a suite of instruments for evaluating usability for academic digital libraries. The model examines effectiveness, efficiency, satisfaction, and learnability. It is found that there exists an interlocking relationship among effectiveness, efficiency, and satisfaction. It also examines how learnability interacts with these three attributes.
... With a DLI the user can search for images, maps or other environmental data and metadata via a cartographic front-end on the Internet. The best known digital library of spatial data is the Alexandria Digital Library (Buttenfield, 1999). However, like all powerful tools, geographers need to recognize the implicit power-knowledge structures, and how they may be used to cross-match and cross-reference data on individuals (Goss, 1995). ...
Two developments in cartography mark an epistemic break with the assumption that maps are unproblematic communication devices. These are 1) investigations of maps as practices of power-knowledge; and 2) 'geographic visualization' (GVis) which uses the map's power to explore, analyze and visualize spatial datasets to understand patterns better. These developments are key components of a 'maps as social constructions' approach, emphasizing the genealogy of power in mapping practices, and enabling multiple, contingent and exploratory perspectives of data. Furthermore, this approach is an opportunity for cartography to renew its relationship with a critical human geography.
The work presents study on coordinated and multiple views (CMV) geovisualisation tools, i.e., interactive tools consisting of several views presenting spatial data in various forms, including maps. The study covers both theoretical considerations and eye-tracking empirical study. Among other things, the author aimed to explore how inexperienced users learn and work with CMV geovisualization tools. On this basis, guidelines were formulated for CMV designers and authors of training materials supporting these tools.
The contribution deals with methodological issues connected with the evaluation of cartographic materials. A new research design for map evaluation comes from an extensive cooperation between cartographers and psychologists. Contrary to strictly quantitative approaches, we propose a mixed research design, combining the quantitative and qualitative methods, because also the qualitative exploration of the corresponding cognitive strategies is in some cases necessary for an in-depth interpretation of the results. Proposed research design is demonstrated by a sample experiment concerning comparison and evaluation of suitability of two different cartographic approaches to avalanche hazard visualization. The quantitative analysis enables to detect potential differences in the compared visualization methods in the sense of their effectiveness and efficiency, and the qualitative analysis of the eye-movement data might improve our understanding of the ways different cognitive strategies are applied. The experiment was carried on the newly developed interactive assessment tool MUTEP, which substantially increases the reliability of experiments in cognitive cartography. MUTEP enables easy data collection and effective evaluation of the results which could be subsequently analyzed.
Universities are investing heavily in electronic resources. As a way of embracing new developments, the University of Ilorin, Nigeria, has spent millions of dollars building a usable e-library. However, research indicates that potential users may still not use e-libraries. This study examines user acceptance of e-library from the perspective of technology acceptance mode (TAM). E-library system characteristics, organisational context, and individual characteristics are identified as variables that determine acceptance. Data was collected through self-designed questionnaire from 1,500 undergraduate users of the e-library. The findings revealed that the acceptance constructs, ease of use, perceived usefulness, actual use, satisfaction, relevance, awareness, computer/internet self-efficacy, and social influence, significantly correlate with e-library acceptance. The study suggests that all eight factors jointly pulled 69% prediction of the users’ acceptance of e-library. The study recommends that e-library users at the university increase their computer and internet self-efficacy, which significantly enhances their use of the e-library system. The university can assist in this matter by organising computer training for the students.
This chapter presents different aspects of digital library interface design and evaluation. It starts from the identification of unique user needs in digital library use. The chapter illustrates the iterative process in the design and implementation of a user interface from the conceptual design, prototype design, and customized design to usability testing. Usability testing is a key focus of the chapter, in particular, usability testing constructs and criteria. Help system design is another essential component for interface design, which requires an understanding of help-seeking situations in order to develop help mechanisms to best facilitate users’ effective interaction with digital libraries. Finally, the issue to design digital library interfaces for people with disabilities is also addressed.
In view of the important role Information Communication and Technology play in education, most administrators of higher educational institutions have invested into these tools to advance teaching, learning and research. In spite of these huge investments, students, lecturers and administrators are yet to fully adopt these tools. This paper seeks to review obstacles which inhibit adoption of ICT in higher educational institutions. The review identified lack of institutional support, financial support, time to learn new technologies, access to computing, technical support and training. Other obstacles identified were technology reliability, uncertainty about its worth, resistance to change, negative attitude, awareness, relevance, ease of use, attitude of organization, and computer literacy. The article concluded that knowing the extent to which these barriers affect individuals and institutions may help in taking decision on how to tackle them.
The intent of this chapter is to identify constructs and criteria for digital library (DL) evaluation based on document analysis. Eighty-five relevant articles and five websites were reviewed to generate the evaluation constructs and criteria. The findings consist of ten constructs, including: collection, information organization, interface design, system performance, effects on users, user engagement, services, preservation, sustainability/administration, and context of use with associated criteria for each dimension. In addition, this paper discusses challenges in DL evaluation research and practices.
This chapter focuses on the usability evaluation of Digital Libraries (DL). The research literature in this area is reviewed. The strengths and weaknesses of the traditional evaluation techniques are discussed. A Cognitive Task Analysis (CTA) approach is proposed based on its applicability for the iterative and formative usability evaluation of digital libraries. The theoretical framework and features of CTA are explained. The advantages of the proposed approach for the usability evaluation of digital libraries are elucidated and the procedure of carrying out this type of evaluation is detailed. Future directions for DL usability evaluation, especially for the purpose of iterative design process, are recommended.
The present paper discusses various digital information services and the methodology to design and develop the services using the potential of digital libraries. It also highlights the issues, evaluation of digital libraries.
This is a report on ongoing work on a digital library service focusing on user experience design. The service examined is the Finnish Digital Library, also called Finna. The service is being developed in collaboration between Finnish libraries, archives, and museums, which brings challenges and opportunities to the design of user experience. The report describes the current organizational settings of the design process by listing its actors, activities, and aims.
The current increase in the publication of "Open Access" data has greatly facilitated the emergence of initiatives to provide users with information from different areas of expertise. Initiatives like DBPedia have attracted the interest from several organizations and entities involved in the hosting, processing and display of large amounts of data, willing to openly share them. However, the vast amount of information presented today hinders the location of resources, making it necessary to devise interfaces that facilitate this task, taking into account the peculiarities of the technology. The following article presents the results of the evaluation of a visual interface prototype that facilitates the search, display and visualization of organizations classified by their domain in the Linking Open Data (LOD) cloud
The author is deeply grateful to Petra Galle and Michael Kluck as well as to all who participated at evaluating IREON. The author is also indebted to Lacey Prpic Hedtke and Stefanie Schweller.
The emergence of the Internet allows millions of people to use a variety of electronic information retrieval systems, such as: digital libraries, Web search engines, online databases, and online public access catalogs. Interactive Information Retrieval in Digital Environments provides theoretical framework in understanding the nature of information retrieval, and offers implications for the design and evolution of interactive information retrieval systems. Interactive Information Retrieval in Digital Environments includes the integration of existing frameworks on user-oriented information retrieval systems across multiple disciplines; the comprehensive review of empirical studies of interactive information retrieval systems for different types of users, tasks, and subtasks; and the discussion of how to evaluate interactive information retrieval systems. Researchers, designers, teachers, scholars, and professionals will gain the foundation for new research on this subject matter, and guidance to evaluate new information retrieval systems for the general public as well as for specific user groups.
As computer usage continues to grow exponentially, the desire of users to use electronic publications (e-publications) has also increased tremendously. This has led to the publication of materials in electronic form as e-publications on both CD-ROMs and web. The electronic book (e-book) is one of the several forms of e-publications and its popularity has been growing steadily for the past decade. This bibliography provides all references of the published literature on e-books and related technologies in one source so that it will save time for others in conducting literature searches and reviewing the developments. The information included in this bibliography is collected systematically from all the published sources in the world such as journal articles, conference papers, conference proceedings, books, reports and PhD theses on e-books until the last quarter of 2011. Mainly, it covers e-books, e-books publishing, the impact of e-books on different types of users, e-book publishing techniques and trends, e-book user interfaces and other technologies related to e-publications. This bibliography will be useful to all researchers conducting research in any areas related to e-books and e-book publishing.
The development of widespread capabilities for electronic archival and dissemination of data can be coupled with advances in information systems technology to deliver large volumes of information very fast. Paradoxically, as greater volumes of information become available on electronic information networks, they become increasingly difficult to access. Nowhere is this situation more pressing than in the case of spatial information, which has been traditionally treated as a 'separate' problem by archivists, due to complexities of spatial ordering and indexing. A research project recently funded by NSF will address these problems and implement a working digital library testbed over the next four years. This paper will focus upon one aspect of the testbed, namely evaluating user requirements to inform interface design. The paper presents the evaluation plan, using hypermedia tools to collect real-time interactive logs of user activities on the testbed under design. Conventionally, interactive logging is analyzed by deterministic measures of performance such as counting keystrokes. In this project, the interactive log data will be analyzed using protocol analysis, which has been shown to provide a rich source of information to formalize understanding about semi-structured and intuitive knowledge.
We offer an introduction to contextual design as an emergent method for building effective systems. Contextual design addresses a number of the inadequacies in previous methods by emphasizing: interview methods conducted in the context of the user's work, codesigning with the user, building an understanding of work in context, and summarizing conclusions through out the research. We contrast this design method to usability engineering and artifact examination.
We analyze a large-scale custom software effort, the Worm Community System (WCS), a collaborative system designed for a geographically dispersed community of geneticists. There were complex challenges in creating this infrastructural tool, ranging from simple lack of resources to complex organizational and intellectual communication failures and tradeoffs. Despite high user satisfaction with the system and interface, and extensive user needs assessment, feedback, and analysis, many users experienced difficulties in signing on and use. The study was conducted during a time of unprecedented growth in the Internet and its utilities (1991-1994), and many respondents turned to the World Wide Web for their information exchange. Using Bateson's model of levels of learning, we analyze the levels of infrastructural complexity involved in system access and designer-user communication. We analyze the connection between systems development aimed at supporting specific forms of collaborative knowledge work, local organizational transformation, and large-scale infrastructural change.
The value of theoretical analyses in user interface design has been hotly debated. All sides agree that it is difficult to apply current theoretical models within the constraints of real-world development projects. We attack this problem in the context of bringing the theoretical ideas within a model of exploratory learning [19] to bear on the evaluation of alternative interfaces for walk-up-and-use systems. We derived a “cognitive walkthrough” procedure for systematically evaluating features of an interface in the context of the theory. Four people independently applied this procedure to four alternative interfaces for which we have empirical usability data. Consideration of the walkthrough sheds light on the consistency with which such a procedure can be applied as well as the accuracy of the results.
Human-System interaction has been and will continue to be of interest to many researchers of various disciplines: engineers, computer scientists, psychologists, and social scientists. The research in Human-System Interaction (HSI) has progressed from the era of using anthropomorphic data to design workspace to the current period which utilizes human and artificial sensors to design sensory-based cooperative workspace. In either of these developments, HSI has been known to be complex. In 1994, we initiated a series of symposiums on Human Interaction with Complex Systems. It was then that various ideas surrounding HSI for today and tomorrow were discussed by many scientists in the related disciplines. As a follow-up, in 1995 the Second Symposium was organized. The objective of this symposium was to attempt to defme a framework, principles, and theories for HSI research. This book is the result of that symposium. The 1995 symposium brought together a number of experts in the area of HSI. The symposium was more focused on expert opinions and testimonies than traditional meetings for technical papers. There were three reasons for that approach.
Scientific work is heterogeneous, requiring many different actors and viewpoints. It also requires cooperation. The two create tension between divergent viewpoints and the need for generalizable findings. We present a model of how one group of actors managed this tension. It draws on the work of amateurs, professionals, administrators and others connected to the Museum of Vertebrate Zoology at the University of California, Berkeley, during its early years. Extending the Latour-Callon model of interessement, two major activities are central for translating between viewpoints: standardization of methods, and the development of `boundary objects'. Boundary objects are both adaptable to different viewpoints and robust enough to maintain identity across them. We distinguish four types of boundary objects: repositories, ideal types, coincident boundaries and standardized forms.
A study involving 12 genuine reference questions and 24 UK public libraries was undertaken. Using a team of volunteers to apply questions of a range of difficulty and complexity, responses were gathered and analysed in terms of accuracy, completeness and timeliness. Results showed the personal skills of respondents had more impact on results that official policies and procedures.
A post-positivist research approach advocates methodological pluralism. It is based on the assumption that the method to be applied in a particular study should be selected based on the research question being addressed. The research described here takes a post-positivist approach, applying interpretive research in two ways: in an exploratory study of end-user computing conducted prior to a positivist study and in a study of end-user searching behaviors conducted concurrently with a positivist study.
Our work embodies one approach to developing theories of use that can be embedded in programs and new ways of working. We argue that building digital libraries that work will require that systems design, development and deployment be grounded in use and done collaboratively with users, and we will describe our approach to accomplishing this as well as the issues and questions it raises.
Describes tracking tools used by designers and users to evaluate the efficacy of hypermedia systems. Highlights include human-computer interaction research; tracking tools and user-centered design; and three examples from the Interactive Multimedia Group at Cornell University that illustrate uses of various tracking tools. (27 references) (LRW)
As computer-based information retrieval and communication systems become more commonplace, researchers have a greater opportunity to evaluate the uses and impacts of new communication technologies. The systems and the kinds of data now available are discussed along with advantages and disadvantages of using computer-monitored data.
The paper summarizes recent work on organizations, artificial intelligence systems, human-computer interaction, etc., which emphasizes thesituated, distributed, andfluid nature of social systems. This contrasts with the traditional way of writing and thinking about social systems which sees them as disembodied, ideal, formal notions of thought. The implications of this new view of systems for social actors, information, knowledge, and technology are discussed. The literature reviewed offers a new way of talking about systems and their practices.
The paper presents a background survey of the existing state-of-the-art as it relates to monitoring of information systems. It addresses both historical and current approaches and both manual and automated techniques. The general concept of automated monitoring into a well-defined methodology, categorizing the generic uses for monitoring, identifying specific objectives of monitoring and translating these objectives into detailed parameters are developed. Methodologies, techniques and theoretical foundations essential for analyzing monitored data are formulated. Desirable computer-based support requirements for data analysis also are discussed. Conclusions and implications for future research and development efforts in monitoring and evaluation of on-line information systems are highlighted.
This chapter examines how work with computer-based and networked systems can be organized in such a way as to have an effect on the quality of working life for clerical workers, administrative staff, professionals, and managers. It discusses social design of computer and communications systems to help indicate how technologies are shaped by organizations as part of a computerization effort. The consequences of computing or networking for work and workers are a byproduct of socio-technical configurations, or social designs, rather than of technology alone. The chapter examines four common conceptual models of organizational behavior: combinations of rational and natural, and open and closed systems. It conceptually integrates the analysis of workplaces that employ more traditional computing arrangements and those that use computerized networks and communication systems. It examines the study of computerization and work life within an explicit set of models drawn from sociological theories of organization and emphasizes on the tacit conceptions of organizational behavior.
Several measures—such as recall, precision, term overlap, and efficiency— have been used to evaluate bibliographic searching. When applied to searches for specific facts in a full-text database, these measures seem less appropriate. For instance, recall is reduced to a binary measure reflecting the success or failure of the search to retrieve the desired fact; and lack of precision may simply reflect the searcher's unwillingness to expend further effort in narrowing a search. It is likely that new measures will need to be developed, and the applicability of known measures to factual searches in full-text databases needs to be evaluated. A study was conducted to evaluate 21 measures of performance on factual searches of a full-text database. The measures included two measures of recall, two measures of precision, seven measures of search term overlap, seven measures of improvement in search term overlap, and three measures of efficiency. Each of these measures was calculated for the searches performed by each of 26 first-year medical students on INQUIRER, a database of facts and concepts in microbiology. Their reliability and construct validity were investigated. Their underlying structure consisted of three factors: Process/Outcome (precision, recall, and term overlap). Improvement in Term Overlap, and Efficiency. One scale for each factor was constructed after eliminating 7 of the original 21 variables. Each of these scales demonstrated adequate reliability for research purposes. The utility of these measures in future research on online searching is discussed.
This paper explores the relevance of recent feminist reconstructions of objectivity for the development of alternative visions of technology production and use. I take as my starting place the working relations that make up the design and use of technical systems. Working relations are understood as networks or webs of connections that sustain the visible and invisible work required to construct coherent technologies and put them into use. I outline the boundaries that characterize current relations of development and use, and the boundary crossings required to transform them. Three contrasting premises for design-the view from nowhere, detached engagement, and located accountability — are taken to represent incommensurate alternatives for a politics of professional design. From the position of located accountability, I close by sketching aspects of what a feminist politics and associated practices of system development could be.
This paper presents a new methodology for performing theory-based evaluations of user interface designs early in the design cycle. The methodology is an adaptation of the design walkthrough techniques that have been used for many years in the software engineering community. Traditional walkthroughs involve hand simulation of sections of code to ensure that they implement specified functionality. The method we present involves hand simulation of the cognitive activities of a user, to ensure that the user can easily learn to perform tasks that the system is intended to support. The cognitive walkthrough methodology, described in detail, is based on a theory of learning by exploration presented in this paper. There is a summary of preliminary results of effectiveness and comparisons with other design methods.
Usability inspection is the generic name for a set of costeffective ways of evaluating user interfaces to find usability problems. They are fairly informal methods and easy to use.
A pluralistic approach, design computers from the user's point of view
In a virtual world, we are inside an environment of pure information that we can see, hear, and touch. The technology itself is invisible, and carefully adapted to human activity so that we can behave naturally in this artificial world. We can create any imaginable environment and we can experience entirely new perspectives and capabilities within it. A virtual world can be informative, useful, and fun; it can also be boring and uncomfortable. The difference is in the design. The platform and the interactive devices we use, the software tools and the purpose of the environment are all elements in the design of virtual worlds. But the most important component in designing comfortable, functional worlds is the person inside them. Cyberspace technology couples the functions of the computer with human capabilities. This requires that we tailor the technology to people, and refine the fit to individuals. We then have customized interaction with personalized forms of information that can amplify our individual intelligence and broaden our experience. Designing virtual worlds is a challenging departure from traditional interface design. In the first section of this chapter I differentiate between paradigms for screen-based interface design and paradigms for creating virtual worlds. The engineer, the designer, and the participant co-create cyberspace. Each role carries its own set of goals and expectations, its own model of the technology's salient features. In the second section of the chapter I address these multiple perspectives, and how they interrelate in the cooperative design process. In conclusion, I consider broader design issues, including control, politics, and emergent phenomena in cyberspace.
Using the Tltink Aloud Metl~od it1 Cognitive Interface Design Designing for Error
- C Lewis
- Thomas J Ibm
- Watson
- Yorktown Center
- Ny Heights
- C Lewis
- D Norman
Lewis, C. 1982. Using the Tltink Aloud Metl~od it1 Cognitive Interface Design. I B M Research Report RC 9265 (#40713) IBM, Thomas J. Watson Research Center, Yorktown Heights, NY. Lewis, C. and Norman, D. 1986. Designing for Error. In: Norman, D. and Draper, S. (eds.) User Centered System Design: New Perspectives on Human-Computer Itlteraction. Hillsdale, N.J.: Erlbaum.
Tlze Art ofHuman-Comprrter Inlerface Design
- B Laurel
Laurel, B. 1990. Tlze Art ofHuman-Comprrter Inlerface Design. Reading, Massachu-setts: Addison-Wesley.
If You Wattt to Evaluate Your Library
- F W Lancaster
Lancaster, F.W. 1993. If You Wattt to Evaluate Your Library... Champaign: Univer-sity of Illinois Press.
Dilemmas in Cooperative Design
- R Markusson
Markusson, R. 1994. Dilemmas in Cooperative Design. In: Trigg, R., Anderson, S.L. and Dykstra-Ericson, E.A. (eds.) Proceedings Participatory Design Conference, Chapel Hill, N.C., 27-28 Oclober, 1994.
Social Worlds of Knowledge Work: Why Researchers Fail to Effec-tively Use Digital Lihraries. The Information Society, voI
- L Covi
Covi, L. 1996. Social Worlds of Knowledge Work: Why Researchers Fail to Effec-tively Use Digital Lihraries. The Information Society, voI.ll(4): 261-271.
Prirtciplrs ofDesign
- W H Mayall
Mayall, W.H. 1979. Prirtciplrs ofDesign. New York: Van Nostrand Reinhold. Downloaded by [University of North Texas] at 03:37 22 November 2014
Technological Change and Professional Identity New Information Technologies-New Opportunities
- B Nielson
Nielson, B. 1981. Technological Change and Professional Identity. I n Smith, L.C. (ed.) New Information Technologies-New Opportunities. 18th Clinic on Library Applications of Data Processing, Champaign-Urbana, 26-29 April 1981: 101-113.
Geospatial Information Needs of the General Public: Text, Maps and Users" Tasks Geographic Itlformation Sys-te~ns and Libraries: Patroru, Maps and Spatial I~~formation
- M Gluck
Gluck, M. 1996. Geospatial Information Needs of the General Public: Text, Maps and Users" Tasks. In: Smith, L.C. and Gluck, M. Geographic Itlformation Sys-te~ns and Libraries: Patroru, Maps and Spatial I~~formation. Champaign, Illinois:
Usability Itzspectiott Methods
- R L Mack
and Mack, R.L. (eds.) Usability Itzspectiott Methods. New York: John Wiley, 203-233.
A Library for Computers and the World of the Future
- J G A D Kemeny
Kemeny, J.G. 1965. A Library for 2000 A.D. In: Greenberger, M. (ed.) Computers and the World of the Future. Cambridge, Mass: M I T Press, pp. 135-162.
Delivering Maps to the Information Society: A Digital Library for Cartographic Data
- B P Buttenfield
Buttenfield, B.P. 1997. Delivering Maps to the Information Society: A Digital Li-brary for Cartographic Data. Proceedings, 17th Conference of the Internatio~ial Cartographic Association, June 1997, Stockholm, Sweden: 1409-1416.
Online Monitoring Methods for User-Based Design and Evalu-ation for Digital Libraries
- C L Borgman
Borgman, C.L. 1995. Online Monitoring Methods for User-Based Design and Evalu-ation for Digital Libraries. Proceedings, 37th Allerton Institute, Graduate School of Library and Information Science, University of Illinois, Champaign-Urbana, 29-31 October 1995 (no page numbers).
Personal Email Communication with the Alexandria Digital Library Videotape Archivist
- M R Rae
Rae, M.R 1995. Personal Email Communication with the Alexandria Digital Library Videotape Archivist.
The Second Alexandria Design Review: Report on the Workshop On the Web at http:Nwww.alexandria.sdc.ucsb The Library is a Growing Organism in the Era of Online Public Access Catalogues and Communication Networks New Trends in lnterrzational Librariattshlp
- B P Buttenfield
- L Hill
- P A Cochrane
Buttenfield, B.P. and Hill, L. 1997. The Second Alexandria Design Review: Report on the Workshop. Santa Barbara, California, 19-21 February 1997. On the Web at http:Nwww.alexandria.sdc.ucsb.edu/public~documents/papers/adr2 Cochrane, P. A. 1984. The Library is a Growing Organism in the Era of Online Public Access Catalogues and Communication Networks. In: Mohanrajan, P.A. (ed.) New Trends in lnterrzational Librariattshlp. New Delhi: Allied Publishers: 199-206.
Using Scenarios to Develop User-Friendly Videotex Systems
- J Nielson
Nielson, J. 1987. Using Scenarios to Develop User-Friendly Videotex Systems. Pro-ceedirtgs, NordDATAY87 Joint Scandinavian Computer Conference, Trondheim, Norway 15-18 June 1987: 133-138.