Article

Using lag-sequential analysis for understanding interaction sequences in visualizations

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

The investigation of how users make sense of the data provided by information systems is very important for Human Computer Interaction. In this context, understanding the interaction processes of users plays an important role. The analysis of interaction sequences, for example, can provide a deeper understanding about how users solve problems. In this paper we present an analysis of sequences of interactions within a visualization system and compare the results to previous research. We used log file analysis and thinking aloud as methods. There is some indication based on log file analysis that there are interaction patterns which can be generalized. Thinking aloud indicates that some cognitive processes occur together with a higher probability than others.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Lag-sequential analysis is widely used in behavioral research and human computer interaction studies (Sanderson & Fisher, 1994). It is an effective method for identifying patterns and relationships in event-based sequential data (Cuomo, 1994;Pohl, Wallner, & Kriglstein, 2016) and is useful for gaining information about how users solve problems (Pohl et al., 2016). The term "lag" refers to the position of coded events, relative to a target code. ...
... Lag-sequential analysis is widely used in behavioral research and human computer interaction studies (Sanderson & Fisher, 1994). It is an effective method for identifying patterns and relationships in event-based sequential data (Cuomo, 1994;Pohl, Wallner, & Kriglstein, 2016) and is useful for gaining information about how users solve problems (Pohl et al., 2016). The term "lag" refers to the position of coded events, relative to a target code. ...
... The term "lag" refers to the position of coded events, relative to a target code. Lag can be specified in terms of "state," which is used to generate the probability that codes occur at various positions after a target code (Pohl et al., 2016). A limitation of state lag-sequential analysis is that it recognizes only a singular adjacent action and can, therefore, only represent singular sequential relationships (Cuomo, 1994;Sanderson & Fisher, 1994). ...
Article
Full-text available
Objective This research investigates security screeners’ knowledge and the effect that differences in knowledge have on the performance of problem-solving activities. We argue that the development of problem-solving knowledge enables security screeners to perform effective problem-solving activity, which assists search and decision-making processes. Background Airport security screening research has investigated the many variables that affect security screeners’ search and decision making during simulated threat-detection tasks. Although search and decision making are essential aspects of security screening, few studies have investigated the problem-solving knowledge and activities that support security screening task performance. Method Sixteen more-experienced and 24 less-experienced security screeners were observed as they performed x-ray screening in the field at an Australian international airport’s departure security checkpoint. Participants wore eye-tracking glasses and delivered concurrent verbal protocol. Results When interacting with other security screeners, more-experienced screeners demonstrated situational knowledge more than less-experienced screeners, whereas less-experienced screeners experienced more insufficient knowledge. Lag-sequential analysis using combined data from both screener groups showed that situational knowledge facilitated effective problem-solving activity to support search and decision making. Insufficient knowledge led screeners to seek assistance and defer decision making. Conclusion This study expands current understandings of airport security screening. It demonstrates that security screeners develop knowledge that is specific to problem solving. This knowledge assists effective problem-solving activity to support search and decision making, and to mitigate uncertainty during the x-ray screening task. Application Findings can inform future security screening processes, screener training, and technology support tools. Furthermore, findings are potentially transferable to other domains.
... Adjusted residuals of each transition are calculated to determine if the transitional probabilities deviated significantly from the expected value. LSA has shown to be useful for understanding human computer interaction behaviors (e.g., Pohl et al. 2016). For example, in the research on interaction processes with visualization systems, Pohl et al. (2016) found interactions patterns and cognitive processes that occur with a higher probability than others by conducting LSA and indicated the findings of interaction processes can be used to make inferences about users' reasoning processes. ...
... LSA has shown to be useful for understanding human computer interaction behaviors (e.g., Pohl et al. 2016). For example, in the research on interaction processes with visualization systems, Pohl et al. (2016) found interactions patterns and cognitive processes that occur with a higher probability than others by conducting LSA and indicated the findings of interaction processes can be used to make inferences about users' reasoning processes. Chung and Baker (2003) performed LSA using users' logged actions in an interactive learning environment and found these sequential actions can be used as a measure of problem-solving processes. ...
Article
Full-text available
An open-ended serious game can engage students’ scientific problem-solving processes. However, understanding how students learn higher-order thinking skills through solving a problem in an open-ended game system is a challenge. The complex game systems may make learning more difficult for students with different characteristics such as an at-risk label. Recent research stresses the importance of using gameplay data to better understand diverse individuals’ learning behaviors and performances in the context of serious games. We analyzed gameplay data of a serious game called Alien Rescue to identify navigation behavior patterns between at-risk and non-at-risk middle school students. Particularly, we incorporated the combination of lag sequential analysis and sequential pattern mining in statistical analyses. The results revealed that the non-at-risk and at-risk students used problem-solving strategies differently when they navigated the environment. The findings using this integrated method confirmed that additional support was needed for at-risk students in order for them to develop contextual and procedural knowledge for problem-solving in the game environment. The findings provide methodological guidelines for researchers considering a sequential analysis as well as offer practical guidelines for game designers to consider when designing serious games with a complex problem so as to help at-risk students.
... For example, 'Gestalt principles of visual perception' which have their roots in phycology, are widely used by practitioners in designing successful information visualizations (Olshannikova, Ometov, Koucheryavy and Olsson, 2015). Likewise, from the psychological perspective, research shows that users tend to engage in activities resembling natural behavior in everyday life (Pohl, Wallner and Kriglstein, 2016). To this end, the most effective visualization is the one that uses multiple criteria resembling users' everyday life. ...
... To this end, the most effective visualization is the one that uses multiple criteria resembling users' everyday life. Otherwise, too many colors, shapes, and interconnections may cause difficulties in the data comprehension (Pohl et al. 2016). ...
Conference Paper
Companies nowadays use various business intelligence and analytics (BI&A) platforms to support de-cision making. Key challenges for users are to efficiently interact with BI&A platforms and quickly comprehend the insights. The research field that studies user-centric design of systems is Human-Com-puter Interaction (HCI). A lot of research has been conducted to define HCI principles and to design evaluation models in Information Systems (IS). However, BI&A platforms seem to differ from other IS due to information overload and massive data visualization. These factors are making the evaluation of HCI principles in BI&A platform more complex. In this research in progress paper, our goal is to study HCI in BI&A platforms. For this purpose, we redesigned the interface of a retail BI&A platform, main-taining the same amount of the displayed information. In a lab experiment, business users evaluated both versions according to three HCI elements i.e. usability, user experience and aesthetics. During the evaluation, we recorded their expressions, errors and runtime. Short interviews aided us in identifying which factors, associated with these three HCI elements, mostly affect the ability of users to effectively interact with the platform. Closing, we also studied what interface improvements impact the compre-hension of the visualized information.
... Based on this work a more recent investigation has been published in Pohl et al. [33]. In this work, sequences of interactions [34] are studied to get a more comprehensive picture of how users interact with visualizations. The results of these studies will be discussed in more detail later on. ...
Article
Full-text available
Making sense of visualizations is often an open and explorative process. This process is still not very well understood. On the one hand, it is an open question which theoretical models are appropriate for the explanation of these activities. Heuristics and theories of everyday thinking probably describe this process better than more formal models. On the other hand, there are only few detailed investigations of interaction processes with information visualizations. We will try to relate approaches describing the usage of heuristics and everyday thinking with existing empirical studies describing sense-making of visualizations.
... The residuals indicate whether the observed frequency of the IBL transition pattern deviates from its expected value; for statistically significant IBL transition patterns, z > 1.96 (p < 0.05). As the number of tallies affects the z-scores, the use of another index in conjunction with the z-scores is recommended when analysing differences between conditions (McComas et al., 2009;Pohl, Wallner, & Kriglstein, 2016). Thus, we also calculated Yule's Q values for the IBL transition patterns (Bakeman & Gottman, 1997). ...
Thesis
Full-text available
Computer-supported collaborative learning (CSCL) frequently takes the form of inquiry-based learning (IBL) in science education. To achieve the benefits of computer-supported collaborative inquiry-based learning (CSCIL), various scaffolds have been studied from the perspective of what (not how) learning occurs and what (not how) differences emerge between the scaffolded and non-scaffolded conditions. To better address the how questions, my theoretical aim was to develop a temporal analysis procedure for CSCL. Based on a systematic literature review of 78 journal papers, I defined six key operations for the analysis of CSCL’s temporal aspects: proposing research aims regarding the temporal aspects, setting up the context, collecting process data, conceptualising events, conducting temporal analysis methods and interpreting the outcomes. A study of how the included papers performed these operations showed how the researchers implicitly conceptualised the temporal aspects of CSCL when focusing on the characteristics of or interrelations between events over time. My methodological aim was to advance temporal analysis methods to study CSCIL. My empirical aim was to design scaffolds and analyse their role in CSCIL by employing the key operations and advanced methods when groups used a numerical problem-solving tool (Python program) to inquire in undergraduate physics courses. To study how CSCIL occurs, I used video data and visualised the transitions between the IBL phases (i.e. IBL sequences) and groups’ ways of using the Python program for inquiry over time (two groups, n = 10). The identified challenges and productive practices guided the scaffold design. To study how differences emerge between the conditions (46 groups, N = 231), I performed temporal log data analysis (TLDA) and temporal lag sequential analysis (TLSA). Temporal distinctions in how the groups used the Python program between the conditions (captured by TLDA) were associated with the differences in the content and temporal emergence of IBL sequence clusters between the conditions (captured by TLSA of video data). This dissertation demonstrates how temporal analysis may advance our understanding of the premises for successful learning and benefit the design and implementation of scaffolds.
... To examine the behavioral patterns of students, data were collected from the users' logs generated during their learning session. It proved that a log-based approach can help researchers to study behavior sequences and understand in-depth how participants solve problems (Pohl, Wallner, & Kriglstein, 2016). Table 1 shows an example of logs. ...
Article
Enriching e-book systems with some features from social networking sites, such as Facebook, has the potential to affect the learning process for students. In this study, we investigated the behavioral patterns of students learning with a Facebook-based e-book approach. An experimentation was conducted in which data were collected from students’ log and then the lag sequential analysis was adopted to explore their behavioral patterns during the learning session. Experimental results identified the significant behavioral learning sequences and revealed that the behaviors: liking, commenting, and sharing posts with peers were the most significant differences between higher- and lower-engagement students. These findings could not only help students to achieve a higher level of engagement, but could also assist researchers who intend to design effective e-book systems based on social networking sites.
... The residuals indicate whether the observed frequency of the IBL transition pattern deviates from its expected value; for statistically significant IBL transition patterns, z > 1.96 (p < 0.05). As the number of tallies affects the z-scores, the use of another index in conjunction with the z-scores is recommended when analysing differences between conditions (McComas et al., 2009;Pohl, Wallner, & Kriglstein, 2016). Thus, we also calculated Yule's Q values for the IBL transition patterns (Bakeman & Gottman, 1997). ...
Article
This paper contributes to the ongoing discussion about analysing the temporal aspects of learning processes in the educational technology research field. Our main aim was to advance methods for analysing temporal aspects of technology-enhanced learning (TEL) processes by introducing the temporal lag sequential analysis (TLSA) technique and by combining TLSA with temporal log data analysis (TLDA). Our secondary aim was to illustrate the potential of these two analysis techniques to reveal the differences between the face-to-face technology-enhanced collaborative inquiry-based learning (CIBL) processes of three different conditions (non-scaffolded, writing scaffolded and script scaffolded groups). The study involved undergraduate university students (N = 231) in natural sciences. The TLDA was based on timestamps and groups' inputs into a TEL environment, and it focused on the groups' temporal ways of using technological resources. The TLSA was based on screen capture videos and audio recordings of the groups’ CIBL processes, and it focused on the inquiry-based learning (IBL) transition patterns (i.e. the transitions between the different IBL phases) discovered by lag sequential analysis and demonstrated by how the IBL transition patterns temporarily emerged. The TLDA findings demonstrated temporal differences regarding how the groups in the different conditions used the available technological resources. The TLSA findings revealed three temporarily distinct IBL transition pattern clusters whose content and temporal emergence varied depending on the condition. Parallel temporal analysis of the log data and the IBL transition patterns indicated that the use of the technological resources temporarily mediated IBL transition patterns. Specifically, we found advantages similar to those of asynchronous online discussions (think before acting) when face-to-face interaction was enhanced with the writing scaffold. The article concludes with a general discussion of the necessity and potential of temporal analysis.
... Step 7 Finally, the sequences of activities reach significance levels depending on the computation of z-values (that is > z 1.96) and Yule's Q (that is > Q 0.30) (Bakeman and Gottman, 1997;Pohl, Wallner, & Kriglstein, 2016). Based on Table 12 and Table 13, Table 11. ...
Thesis
In this study, we developed a tablet-based application,namely Authentic U-Fraction to assist elementary school students in learningfractions with authentic contextual support. In this study,an experiment was carried out to investigate the effects of the students’ learning behavior toward the fraction learning achievements using Authentic U-Fraction. Three topics of fractions included are fractions concept, fraction simplification, and fraction addition/subtraction. Totally 54 fifth-grade students were assigned and divide into two groups, one experimental group,and one control group. The control group learned fraction using traditional teaching method and paper-based assignment, while the experimental group using Authentic U-Fraction in an authentic context. After the experiment, the result showedthat the experimental group performed better than the control group,especially in understanding fraction and fraction representation. The reason was that by taking the picture, making fraction representation from the picture, and making annotation facilitated students to learn fraction using Authentic U-Fraction. Authentic U-Fraction also helped students to do more practice by themselves and fully explain their solutions, but it does not affect their learning achievement. Because besides students did more practice, they also need to make a complete annotation that includesthree different representation: linguistic, logic mathematics, and visual representative. More students did practice with complete annotation higher their learning achievement. Application of three scaffoldings on students’ peer assessment also revealed that meaningful peer assessment couldcorrelate to the students’ learning achievement that also strengthened by the teacher assessment score that also has a higher correlation to the learning achievement from the assessment of assignment side. A Multiple regression result shown that teacher assessment score could strongly predict the posttest score. Overall, students satisfied with Authentic U-Fraction because it is easy to use, easy to learn, and it facilitatesthe students to learn fraction in an authentic context. In the final, the result of this study contributes essential implications along with conclusion and suggestion for future research.
... Their framework is based on a literature review. Pohl, Wallner, and Kriglstein (2016) and Reda et al. (2014) investigate interaction processes with information visualisations. The latter point out that the emphasis so far has been on the analysis of outcomes rather than on the sense-making process itself. ...
Article
Full-text available
Visualising how social networks evolve is important in intelligence analysis in order to detect and monitor issues, such as emerging crime patterns or rapidly growing groups of offenders. It remains an open research question how this type of information should be presented for visual exploration. To get a sense of how users work with different types of visualisations, we evaluate a matrix and a node-link diagram in a controlled thinking aloud study. We describe the sense-making strategies that users adopted during explorative and realistic tasks. Thereby, we focus on the user behaviour in switching between the two visualisations and propose a set of nine strategies. Based on a qualitative and quantitative content analysis we show which visualisation supports which strategy better. We find that the two visualisations clearly support intelligence tasks and that for some tasks the combined use is more advantageous than the use of an individual visualisation.
... Metrics such as error rate and task completion time are commonly used to measure user performance but these do not provide details in to patterns of use (i.e., sequences of actions or dwell times). Other methods such as lag-sequential analysis (Pohl, Wallner, & Kriglstein, 2016) can be used to see if users deviate from recommended sequence of activities to achieve a goal, or understand how users solve problems with the displays. There are also tools to track website traffic, such as user flow that visualizes the paths users took through a site, from the source, through the various pages, and where along their paths they exited the site (Clifton, 2012). ...
Article
We proposed new metrics based on information entropy to capture characteristics of screen use and switching among information display screens. These metrics reflect differences in the regularity of the time spent on each screen/display or the number of visits to each screen/display. We also provided simulation and case study to demonstrate the sensitivity and usefulness of the metrics.
... Lag sequential analysis, as a widely adopted approach in behavioral science, has been utilized by educational researchers. It is mainly used to analyze temporal behaviors based on interactive stream data collected through online logs (Pohl, Wallner, & Kriglstein, 2016). In the field of education, particularly, this approach could be used to examine if the transition relationship between any two behaviors is statistically significant as well as find significant sequential patterns in different learning groups (Hou, 2012;Chen, 2014;Hou, 2015;Loh, Sheng, & Li, 2015). ...
Chapter
Full-text available
In a private learning environment, each learner’s interactions with course contents are treasured clues for educators to understand individual and collective learning process. To provide educators with evidence-based insights, this chapter intends to adopt sequential analysis method to unfold learning behavioral differences among different groups of students (grade, subject and registration type) in a university Cloud Classroom system. Experimental results indicate that sophomores undertake more learning tasks than other grades. There are significant differences in task-related and self-monitoring behaviors between liberal arts and science learners. Registered learners have higher participation levels than non-registered ones. Meanwhile, a user study aiming at analyze students’ learning feelings indicates that a fraction of students have dishonest behaviors for achieving a good online performance. Finally, this study discusses behavioral ethical issues emerged in Cloud Classroom, which deserve attention of educators for regulating and optimizing online learning process of students.
Article
Massive open online courses (MOOCs) face the ongoing problem of low pass rates. A number of MOOC learners engage in most or all course activities but fail to pass the course. To improve participation and thereby increase successful completion, this study explores the behavioural differences between certificate achievers and explorers that lead to their different learning outcomes. Eleven behaviours were extracted from more than 80,000 behaviour records for 215 certificate achievers and 456 explorers, which were categorized into four types of activities: graded assessment, content-related, course-related, and interactive activities. Random forest classifier, independent-sample t-testing, and lag sequential analysis were used. The results showed six important behaviours that are highly related to certificate achievement. Certificate achievers engaged in more course-related and graded assessment activities during the course, while explorers engaged in more content-related and interactive activities, and graded assessment activities occurring at the end of the course. Compared to explorers, certificate achievers exhibited more bidirectional behaviours in terms of interactive and course-related activities and more repeated behaviours in terms of course-related and graded assessment activities. The study has significant implications for MOOC learners and instructors in improving learning and teaching performance by implementing appropriate behavioural interventions.
Article
Individual differences exist among learners. Among various individual differences, cognitive styles can strongly predict learners’ learning behavior. Therefore, cognitive styles are essential for the design of online learning. There are a variety of cognitive style dimensions and overlaps exist among these dimensions. In particular, Witkin’s field dependence/independence and Pask’s Holism/Serialism share some similarities. To this end, it is necessary to develop a framework to show overlapped behavior between these two cognitive style dimensions. To address this issue, this study used the Lag Sequential Analysis to examine the overlaps between these two cognitive style dimensions from the aspect of online learning behavior. The results from this study indicated that the overlaps mainly appear in comprehensive/local and dynamic/fixed approaches. Based on the findings of this study, we develop a framework that can support the improvement of instruction design so that the needs of different cognitive style groups can be accommodated. Accordingly, this study is an interdisciplinary work, which makes scientific contributions to three communities, i.e. human–computer interaction, digital learning and learning analytic.
Article
Full-text available
We present results from an experiment aimed at using logs of interactions with a visual analytics application to better understand how interactions lead to insight generation. We performed an insight-based user study of a visual analytics application and ran post hoc quantitative analyses of participants' measured insight metrics and interaction logs. The quantitative analyses identified features of interaction that were correlated with insight characteristics, and we confirmed these findings using a qualitative analysis of video captured during the user study. Results of the experiment include design guidelines for the visual analytics application aimed at supporting insight generation. Furthermore, we demonstrated an analysis method using interaction logs that identified which interaction patterns led to insights, going beyond insight-based evaluations that only quantify insight characteristics. We also discuss choices and pitfalls encountered when applying this analysis method, such as the benefits and costs of applying an abstraction framework to application-specific actions before further analysis. Our method can be applied to evaluations of other visualization tools to inform the design of insight-promoting interactions and to better understand analyst behaviors.
Article
Full-text available
Sensemaking is a process of finding meaning from information that often involves activities such as information foraging and hypothesis generation. It can be valuable to maintain a history of the data and reasoning involved. This history, commonly known as provenance information, can be a resource for "reflection-in-action"' during analysis, supporting collaboration between analysts, and can help trace data quality and uncertainty through the analysis process. Currently, there is limited work on utilizing analytic provenance, which captures the interactive data exploration and human reasoning process, to support sensemaking. This article presents and extends the research challenges discussed in a IEEE VIS 2014 workshop on this topic to provide an agenda for sensemaking analytic provenance.
Article
Full-text available
End-users are accustomed to filtering and browsing styled collections of data on professional web sites, but they have few ways to create and publish such information architectures for themselves. This paper presents a full-lifecycle analysis of the Exhibit framework - an end-user tool which provides such functionality - to understand the needs, capabilities, and practices of this class of users. We include interviews, as well as analysis of over 1,800 visualizations and 200,000 web interactions with these visualizations. Our analysis reveals important findings about this user population which generalize to the task of providing better end-user structured content publication tools.
Article
Full-text available
The queries people issue to a search engine and the results clicked following a query change over time. For example, after the earthquake in Japan in March 2011, the query japan spiked in popularity and people issuing the query were more likely to click government-related results than they would prior to the earthquake. We explore the modeling and prediction of such temporal patterns in Web search behavior. We develop a temporal modeling framework adapted from physics and signal processing and harness it to predict temporal patterns in search behavior using smoothing, trends, periodicities, and surprises. Using current and past behavioral data, we develop a learning procedure that can be used to construct models of users' Web search activities. We also develop a novel methodology that learns to select the best prediction model from a family of predictive models for a given query or a class of queries. Experimental results indicate that the predictive models significantly outperform baseline models that weight historical evidence the same for all queries. We present two applications where new methods introduced for the temporal modeling of user behavior significantly improve upon the state of the art. Finally, we discuss opportunities for using models of temporal dynamics to enhance other areas of Web search and information retrieval.
Conference Paper
Full-text available
Distributed cognition and embodiment provide compelling models for how humans think and interact with the environment. Our examination of the use of large, high-resolution displays from an embodied perspective has lead directly to the development of a new sensemaking environment called Analyst's Workspace (AW). AW leverages the embodied resources made more accessible through the physical nature of the display to create a spatial workspace. By combining spatial layout of documents and other artifacts with an entity-centric, explorative investigative approach, AW aims to allow the analyst to externalize elements of the sensemaking process as a part of the investigation, integrated into the visual representations of the data itself. In this paper, we describe the various capabilities of AW and discuss the key principles and concepts underlying its design, emphasizing unique design principles for designing visual analytic tools for large, high-resolution displays.
Article
Full-text available
We describe and demonstrate an extensible framework that supports data exploration and provenance in the context of Human Terrain Analysis (HTA). Working closely with defence analysts we extract requirements and a list of features that characterise data analysed at the end of the HTA chain. From these, we select an appropriate non-classified data source with analogous features, and model it as a set of facets. We develop ProveML, an XML-based extension of the Open Provenance Model, using these facets and augment it with the structures necessary to record the provenance of data, analytical process and interpretations. Through an iterative process, we develop and refine a prototype system for Human Terrain Visual Analytics (HTVA), and demonstrate means of storing, browsing and recalling analytical provenance and process through analytic bookmarks in ProveML. We show how these bookmarks can be combined to form narratives that link back to the live data. Throughout the process, we demonstrate that through structured workshops, rapid prototyping and structured communication with intelligence analysts we are able to establish requirements, and design schema, techniques and tools that meet the requirements of the intelligence community. We use the needs and reactions of defence analysts in defining and steering the methods to validate the framework.
Article
Full-text available
We are quickly passing through the historical moment when people work in front of a single computer, dominated by a small CRT and focused on tasks involving only local information. Networked computers are becoming ubiquitous and are playing increasingly significant roles in our lives and in the basic infrastructures of science, business, and social interaction. For human-computer interaction to advance in the new millennium we need to better understand the emerging dynamic of interaction in which the focus task is no longer confined to the desktop but reaches into a complex networked world of information and computer-mediated interactions. We think the theory of distributed cognition has a special role to play in understanding interactions between people and technologies, for its focus has always been on whole environments: what we really do in them and how we coordinate our activity in them. Distributed cognition provides a radical reorientation of how to think about designing and supporting human-computer interaction. As a theory it is specifically tailored to understanding interactions among people and technologies. In this article we propose distributed cognition as a new foundation for human-computer interaction, sketch an integrated research framework, and use selections from our earlier work to suggest how this framework can provide new opportunities in the design of digital work materials.
Article
Full-text available
Statistical and grammatical techniques are reviewed as an integrated approach to exploratory sequential data analysis (ESDA) for categorical data. The first step is the identification and validation of the categories to be analyzed. The main statistical techniques discussed are log-linear modeling and lag sequential analysis. These methods allow for the statistical evaluation of a wide range of general and specific hypotheses about sequential structure. Grammatical techniques based on definite-clause grammars are described and illustrated, and the complex issue of measuring the goodness of fit of a set of patterns is discussed. Throughout the article, examples from our own research illustrate how the various techniques are used, especially in concert, while carrying out ESDA. In Section 6, several other human-computer interaction and computer-supported cooperative work applications of these techniques are discussed.
Article
Full-text available
Human-computer interaction (HCI) investigators must consider the sequential nature of interaction and must often weigh behavioral, cognitive, and social factors when studying and designing today's increasingly complex systems. In many cases, laboratory experimentation is inappropriate and formal modeling intractable; instead, observational data analysis is frequently the only appropriate empirical approach. Diverse approaches to observational data analysis already exist, which we synthesize as instances of exploratory sequential data analysis (ESDA). In this article, we outline fundamental ESDA characteristics that might help HCI investigators using sequential data make better conceptual and methodological choices. ESDA owes a philosophical debt to exploratory data analysis but focuses on exploring sequential data. Important issues for ESDA are finding an appropriate temporal band for analysis, finding an effective semantics for encoding, and completing an analysis in an acceptable time frame. We survey temporal factors and introduce analysis time:sequence time ratios, which describe the time cost of conducting different types of ESDA. We also introduce the "Eight Cs"-different general transformations that can be performed on sequential data. We conjecture that the Eight Cs, and their combinations, are critical for supporting scientific inference in ESDA. Distinctions are made among three principal ESDA traditions that are relevant for HCI-behavioral, cognitive, and social. We indicate how each ESDA tradition has been used in HCI and describe one technique from each tradition. Last, we outline major practical problems for investigators using observational data and, following our framework, suggest ways such problems might be overcome.
Article
Full-text available
While there have been advances in visualization systems, particularly in multi-view visualizations and visual exploration, the process of building visualizations remains a major bottleneck in data exploration. We show that provenance metadata collected during the creation of pipelines can be reused to suggest similar content in related visualizations and guide semi-automated changes. We introduce the idea of query-by-example in the context of an ensemble of visualizations, and the use of analogies as first-class operations in a system to guide scalable interactions. We describe an implementation of these techniques in VisTrails, a publicly-available, open-source system.
Article
Full-text available
Modern information visualisation systems do not only support interactivity but also increasingly complex problem solving. In this study we compare two interactive information visualisation systems: VisuExplore and Gravi++. By analysing logfiles we were able to identify sets of activities and interaction patterns users followed while working with these systems. These patterns are an indication of strategies users adopt to find solutions. Identifying such patterns may help in improving the design of future information visualisation systems.
Conference Paper
Full-text available
Exploring large network datasets, such as scientific collaboration networks, is challenging because they often contain a large number of nodes and edges in several types and with multiple attributes. Analyses of such networks are often long and complex, and may require several sessions by multiple users. Therefore, it is often difficult for users to recall their own exploration history or share it with others. We introduce GraphTrail, an interactive visualization for analyzing networks through exploration of node and edge aggregates that captures users' interactions and integrates this history directly in the exploration workspace. To facilitate large network analysis, GraphTrail integrates aggregation with familiar charts, drag-and-drop interaction on a canvas, and a novel pivoting mechanism for transitioning between aggregates. Through a three-month field study with a team of archeologists and a qualitative lab study with ten users, we demonstrate the effectiveness of our design and the benefits of integrated exploration history, including analysis comprehension, insight discovery, and exploration recall.
Chapter
Full-text available
Information visualization research is becoming more established, and as a result, it is becoming increasingly important that research in this field is validated. With the general increase in information visualization research there has also been an increase, albeit disproportionately small, in the amount of empirical work directly focused on information visualization. The purpose of this chapter is to increase awareness of empirical research in general, of its relationship to information visualization in particular; to emphasize its importance; and to encourage thoughtful application of a greater variety of evaluative research methodologies in information visualization.
Article
Full-text available
Why do people create extra representations to help them make sense of situations, diagrams, illustrations, instructions and problems? The obvious explanation—external representations save internal memory and computation—is only part of the story. I discuss seven ways external representations enhance cognitive power: they change the cost structure of the inferential landscape; they provide a structure that can serve as a shareable object of thought; they create persistent referents; they facilitate re-representation; they are often a more natural representation of structure than mental representations; they facilitate the computation of more explicit encoding of information; they enable the construction of arbitrarily complex structure; and they lower the cost of controlling thought—they help coordinate thought. Jointly, these functions allow people to think more powerfully with external representations than without. They allow us to think the previously unthinkable. Keywords: External representations-Thinking-Interactivity-Sense making-Cost structure
Conference Paper
Full-text available
In this paper, we discuss the challenges involved in developing an infrastructure to support a new generation of analytic tools for information analysts. The infrastructure provides data for establishing context about what the analyst is doing with the analytic tools, supports an integration environment to allow suites of tools to work together, and supports evaluation of the analytic tools. We discuss the functionality of the Glass Box, the challenges of evaluating adaptive systems including the capture of data for evaluation metrics, and lessons learned from our experiences to date.
Conference Paper
Full-text available
VisTrails is a new system that enables interactive multiple-view visualizations by simplifying the creation and maintenance of visualization pipelines, and by optimizing their execution. It provides a general infrastructure that can be combined with existing visualization systems and libraries. A key component of VisTrails is the visualization trail (vistrail), a formal specification of a pipeline. Unlike existing dataflow-based systems, in VisTrails there is a clear separation between the specification of a pipeline and its execution instances. This separation enables powerful scripting capabilities and provides a scalable mechanism for generating a large number of visualizations. VisTrails also leverages the vistrail specification to identify and avoid redundant operations. This optimization is especially useful while exploring multiple visualizations. When variations of the same pipeline need to be executed, substantial speedups can be obtained by caching the results of overlapping subsequences of the pipelines. In this paper, we describe the design and implementation of VisTrails, and show its effectiveness in different application scenarios.
Conference Paper
Full-text available
As the information being visualized and the process of understanding it both become increasingly complex, it is necessary to develop new visualization approaches that facilitate the flow of human reasoning. In this paper, we endeavor to push visualization design a step beyond current user models by discussing a modeling framework of human "higher cognition." Based on this cognition model, we present design guidelines for the development of visual interfaces designed to maximize the complementary cognitive strengths of both human and computer. Additionally, this paper introduces cognition-based principles of visualization design. Some of these principles are already being reflected in the better visual analytics designs, while others have not yet been applied or fully applied. But none have explained the deeper rationale that the model provides. Lastly, we discuss and assess these visual analytics guidelines through the evaluation of several visualization examples.
Conference Paper
Full-text available
Visual analytics tools provide powerful visual representations in order to support the sense-making process. In this process, analysts typically iterate through sequences of steps many times, varying parameters each time. Few visual analytics tools support this process well, nor do they provide support for visualizing and understanding the analysis process itself. To help analysts understand, explore, reference, and reuse their analysis process, we present a visual analytics system named CzSaw (See-Saw) that provides an editable and re-playable history navigation channel in addition to multiple visual representations of document collections and the entities within them (in a manner inspired by Jigsaw). Conventional history navigation tools range from basic undo and redo to branching timelines of user actions. In CzSaw's approach to this, first, user interactions are translated into a script language that drives the underlying scripting-driven propagation system. The latter allows analysts to edit analysis steps, and ultimately to program them. Second, on this base, we build both a history view showing progress and alternative paths, and a dependency graph showing the underlying logic of the analysis and dependency relations among the results of each step. These tools result in a visual model of the sense-making process, providing a way for analysts to visualize their analysis process, to reinterpret the problem, explore alternative paths, extract analysis patterns from existing history, and reuse them with other related analyses.
Conference Paper
Full-text available
The final product of an analyst's investigation using a visualization is often a report of the discovered knowledge, as well as the methods employed and reasoning behind the discovery. We believe that analysts may have difficulty keeping track of their knowledge discovery process and will require tools to assist in accurately recovering their reasoning. We first report on a study examining analysts' recall of their strategies and methods, demonstrating their lack of memory of the path of knowledge discovery. We then explore whether a tool visualizing the steps of the visual analysis can aid users in recalling their reasoning process. The results of our second study indicate that visualizations of interaction logs can serve as an effective memory aid, allowing analysts to recall additional details of their strategies and decisions.
Article
Full-text available
There is a growing recognition within the visual analytics community that interaction and inquiry are inextricable. It is through the interactive manipulation of a visual interface–the analytic discourse–that knowledge is constructed, tested, refined and shared. This article reflects on the interaction challenges raised in the visual analytics research and development agenda and further explores the relationship between interaction and cognition. It identifies recent exemplars of visual analytics research that have made substantive progress toward the goals of a true science of interaction, which must include theories and testable premises about the most appropriate mechanisms for human–information interaction. Seven areas for further work are highlighted as those among the highest priorities for the next 5 years of visual analytics research: ubiquitous, embodied interaction; capturing user intentionality; knowledge-based interfaces; collaboration; principles of design and perception; interoperability; and interaction evaluation. Ultimately, the goal of a science of interaction is to support the visual analytics and human–computer interaction communities through the recognition and implementation of best practices in the representation and manipulation of visual displays.
Article
Full-text available
The benefits of research tools in exploratory search with user interface capability are discussed. Exploratoty search provide information along with pitfalls and costs and helps in gaining complete knowledge about the product. The tools to support exploratory searching make possible to use use web resources to conduct lookup, learning, and investigate tasks with fluid user interfaces. A good web design supports exploratory search together with client-side applications, that provides people with ways to trade off personal behavior data for added value services. It is recommended that more computational resources should be designed for exploratory search, that will provide easy to apply exploratory search tools and help information seekers in the use of information resources.
Conference Paper
Full-text available
The effectiveness of animation in visualization is an interesting research topic that led to contradicting results in the past. On top of that, we are facing three additional challenges when exploring patient cohorts: irregular sampling, data wear, and data sets covering different portions of time. We present TimeRider, an improved animated scatter plot for cohorts of diabetes patients that tackles these challenges along with its evaluation with physicians. Results show that animation does support physicians in their work and provide further domain-specific evidence in the discussion on the effectiveness of animation. KeywordsInformation Visualization–animation–time–medical data
Article
Full-text available
In this study we investigated the feasibility of a novel user interface to support the measurement of problem-solving processes. Our research questions addressed the use of a "click-through" interface to measure the "generate-and-test" problem-solving process for a design problem. A click-through interface requires the user to explicitly perform an online action (e.g., to view time, the user has to click on a "time" icon). This interface allowed us to measure participants' intentional acts. Freshman college students were given the task of modifying a given, computer-interactive bicycle pump to satisfy performance requirements. The simulation interface provided participants with point-and-click access to controls to modify pump parameters, to run the simulation, to view important information, and to attempt to solve the task. Lag sequential analyses of participants' problem-solving processes over time showed cyclical behavior consistent with the generate-and-test strategy of modifying the pump design, running the simulation, viewing the information, and then either modifying the design or attempting to solve the problem and then modifying the design again. This behavior set was remarkably stable, with most lag 1 associations greater than .80. Our approach to measuring problem-solving processes appears feasible and promising, but more work is needed to gather additional validity evidence.
Article
Full-text available
With visual analytical tools becoming more sophisticated and prevalent in the analysis communities, it is now apparent that understanding how analysts utilize these tools is more important than ever. Such understanding can lead to improving the tools, but a more subtle and equally important aspect lies in the discovery of the analysts ’ reasoning process for solving complex problems through the use of these visual analytical tools. In this paper we demonstrate that we were able to identify several of the strategies, methods, and findings of an analysis process using a financial visual analytical tool through the examination of an analyst’s interaction log. In our study, we recorded the interactions and think-alouds of 10 financial analysts in a fraud detection task. By examining their interaction logs, we are able to quantitatively show that 60 % of strategies, 60% of methods, and 79 % of findings could be recovered through the use of two visual analytic log analysis tools. 1
Article
Full-text available
Building visualization and analysis pipelines is a large hurdle in the adoption of visualization and workflow systems by domain scientists. In this paper, we propose techniques to help users construct pipelines by consensus--automatically suggesting completions based on a database of previously created pipelines. In particular, we compute correspondences between existing pipeline subgraphs from the database, and use these to predict sets of likely pipeline additions to a given partial pipeline. By presenting these predictions in a carefully designed interface, users can create visualizations and other data products more efficiently because they can augment their normal work patterns with the suggested completions. We present an implementation of our technique in a publicly-available, open-source scientific workflow system and demonstrate efficiency gains in real-world situations.
Article
Full-text available
Even though information visualization (InfoVis) research has matured in recent years, it is generally acknowledged that the field still lacks supporting, encompassing theories. In this paper, we argue that the distributed cognition framework can be used to substantiate the theoretical foundation of InfoVis. We highlight fundamental assumptions and theoretical constructs of the distributed cognition approach, based on the cognitive science literature and a real life scenario. We then discuss how the distributed cognition framework can have an impact on the research directions and methodologies we take as InfoVis researchers. Our contributions are as follows. First, we highlight the view that cognition is more an emergent property of interaction than a property of the human mind. Second, we argue that a reductionist approach to study the abstract properties of isolated human minds may not be useful in informing InfoVis design. Finally we propose to make cognition an explicit research agenda, and discuss the implications on how we perform evaluation and theory building.
Article
Visual analytics is inherently a collaboration between human and computer. However, in current visual analytics systems, the computer has limited means of knowing about its users and their analysis processes. While existing research has shown that a user's interactions with a system reflect a large amount of the user's reasoning process, there has been limited advancement in developing automated, real-time techniques that mine interactions to learn about the user. In this paper, we demonstrate that we can accurately predict a user's task performance and infer some user personality traits by using machine learning techniques to analyze interaction data. Specifically, we conduct an experiment in which participants perform a visual search task, and apply well-known machine learning algorithms to three encodings of the users' interaction data. We achieve, depending on algorithm and encoding, between 62% and 83% accuracy at predicting whether each user will be fast or slow at completing the task. Beyond predicting performance, we demonstrate that using the same techniques, we can infer aspects of the user's personality factors, including locus of control, extraversion, and neuroticism. Further analyses show that strong results can be attained with limited observation time: in one case 95% of the final accuracy is gained after a quarter of the average task completion time. Overall, our findings show that interactions can provide information to the computer about its human collaborator, and establish a foundation for realizing mixed-initiative visual analytics systems.
Article
There is an increasing growth of Social Network Services (SNSs). A variety of SNSs are applied in online interpersonal platforms. Among them, asynchronous and synchronous discussions are widely examined. However, there is a lack of research into the effects of integrated discussion services that combines asynchronous and synchronous discussions. Thus, the study investigates users’ performances and behavior patterns in a mixed discussion model that integrates asynchronous and synchronous discussions with a lag sequential analysis. The results showed that most groups chose to adopt the mixed discussion model (i.e., using both synchronous and asynchronous discussions), and only one group totally adopted the asynchronous discussion model. We further analyzed the learners’ learning effectiveness and behavioral patterns, and the results indicated that the groups using the mixed model had a positive performance to a certain extent in terms of learning effectiveness and knowledge construction. In addition, users with the mixed discussion model demonstrated diverse behaviors, which were more complex than that of those with a single-way discussion model. Furthermore, regarding the groups using “balanced synchronous and asynchronous discussions” and the groups “mainly using synchronous discussion supplemented by less asynchronous discussion” in the mixed model (simultaneously using synchronous and asynchronous discussions), those users who mainly used synchronous discussion supplemented by asynchronous discussion had more diverse behaviors of knowledge construction.
Article
Visual analytics employs interactive visualizations to integrate users’ knowledge and inference capability into numerical/algorithmic data analysis processes. It is an active research field that has applications in many sectors, such as security, finance, and business. The growing popularity of visual analytics in recent years creates the need for a broad survey that reviews and assesses the recent developments in the field. This report reviews and classifies recent work into a set of application categories including space and time, multivariate, text, graph and network, and other applications. More importantly, this report presents analytics space, inspired by design space, which relates each application category to the key steps in visual analytics, including visual mapping, model-based analysis, and user interactions. We explore and discuss the analytics space to add the current understanding and better understand research trends in the field.
Article
Our purpose is to bring together perspectives concerning the processing and use of statistical graphs to identify critical factors that appear to influence graph comprehension and to suggest instructional implications. After providing a synthesis of information about the nature and structure of graphs, we define graph comprehension. We consider 4 critical factors that appear to affect graph comprehension: the purposes for using graphs, task characteristics, discipline characteristics, and reader characteristics. A construct called graph sense is defined. A sequence for ordering the introduction of graphs is proposed. We conclude with a discussion of issues involved in making sense of quantitative information using graphs and ways instruction may be modified to promote such sense making.
Conference Paper
In this position paper, we describe two methods for the analysis of sequences of interaction with information visualization tools -- log file analysis and thinking aloud. Such an analysis is valuable because it can help designers to understand cognition processes of the users and, as a consequence, to improve the design of information visualizations. In this context, we also discuss the issue of categorization of user activities. Categorization helps researchers to generalize results and compare different information visualization tools.
Article
Visual analytics emphasizes the interplay between visualization, analytical procedures performed by computers and human perceptual and cognitive activities. Human reasoning is an important element in this context. There are several theories in psychology and HCI explaining open-ended and exploratory reasoning. Five of these theories (sensemaking theories, gestalt theories, distributed cognition, graph comprehension theories and skill-rule-knowledge models) are described in this paper. We discuss their relevance for visual analytics. In order to do this more systematically, we developed a schema of categories relevant for visual analytics research and evaluation. All these theories have strengths but also weaknesses in explaining interaction with visual analytics systems. A possibility to overcome the weaknesses would be to combine two or more of these theories.
Article
The considerable previous work characterizing visualization usage has focused on low-level tasks or interactions and high-level tasks, leaving a gap between them that is not addressed. This gap leads to a lack of distinction between the ends and means of a task, limiting the potential for rigorous analysis. We contribute a multi-level typology of visualization tasks to address this gap, distinguishing why and how a visualization task is performed, as well as what the task inputs and outputs are. Our typology allows complex tasks to be expressed as sequences of interdependent simpler tasks, resulting in concise and flexible descriptions for tasks of varying complexity and scope. It provides abstract rather than domain-specific descriptions of tasks, so that useful comparisons can be made between visualization systems targeted at different application domains. This descriptive power supports a level of analysis required for the generation of new designs, by guiding the translation of domain-specific problems into abstract tasks, and for the qualitative evaluation of visualization usage. We demonstrate the benefits of our approach in a detailed case study, comparing task descriptions from our typology to those derived from related work. We also discuss the similarities and differences between our typology and over two dozen extant classification systems and theoretical frameworks from the literatures of visualization, human-computer interaction, information retrieval, communications, and cartography.
Article
The applicability of sequential data analysis (SDA) techniques for analysing usability test data is examined. SDA techniques include transition matrix analysis, lag sequential analysis, frequency of cycles, graphical summarization techniques, and pattern analysis techniques. A subset of each was used in analysing the data from three usability studies. The encoding schemes used, the analysis routines run, software tools to support encoding and analysis (SHAPA and the Maximal Repeating Pattern analysis tool), and their interactions are discussed. The different types of usability problems which can be extracted from the data when analysed with SDA techniques are illustrated. It is concluded that the SDA techniques will be useful once the state-of-the-art in software support is able to provide the analyst greater flexibility in applying the analysis routines. Without the ability to apply analysis routines to multiple data levels, too much work is involved in obtaining a complete analysis of usability problems at all levels.
Article
One method for identifying contingency relationships in a sequence of observed behaviors studies occurrences of matching behaviors at successive event or time steps from a criterion event. This paper briefly summarizes the method and presents a computer program for generating lag conditional probabilities and associated measures for assessing statistical significance.
Article
Despite the growing number of systems providing visual analytic support for investigative analysis, few empirical studies of the potential benefits of such systems have been conducted, particularly controlled, comparative evaluations. Determining how such systems foster insight and sensemaking is important for their continued growth and study, however. Furthermore, studies that identify how people use such systems and why they benefit (or not) can help inform the design of new systems in this area. We conducted an evaluation of the visual analytics system Jigsaw employed in a small investigative sensemaking exercise, and compared its use to three other more traditional methods of analysis. Sixteen participants performed a simulated intelligence analysis task under one of the four conditions. Experimental results suggest that Jigsaw assisted participants to analyze the data and identify an embedded threat. We describe different analysis strategies used by study participants and how computational support (or the lack thereof) influenced the strategies. We then illustrate several characteristics of the sensemaking process identified in the study and provide design implications for investigative analysis tools based thereon. We conclude with recommendations on metrics and techniques for evaluating visual analytics systems for investigative analysis.
Article
We present a study of complex visualization usage by expert meteorological forecasters. We performed a protocol analysis and examined the types of visualizations they examined. We present evidence for how experts are able to make use of complex visualizations. Our findings suggest that users of complex visualizations create qualitative mental models from which they can then generate quantitative information. In order to build their qualitative mental models, forecasters integrated information across multiple visualizations and extracted primarily qualitative information from visualizations in a goal-directed manner. We discuss both theoretical and practical implications of this study.
Article
We present data and argument to show that in Tetris—a real-time, interactive video game-certain cognitive and perceptual problems are more quicklv, easily, and reliably solved by performing actions in the world than by performing computational actions in the head alone. We have found that some of the translations and rotations made by players of this video game are best understood as actions that use the world to improve cognition. These actions are not used to implement a plan, or to implement a reaction; they are used to change the world in order to simplify the problem-solving task. Thus, we distinguish pragmatic actions—actions performed to bring one physically closer to a goal—from epistemic actions —actions performed to uncover informatioan that is hidden or hard to compute mentally.To illustrate the need for epistemic actions, we first develop a standard information-processing model of Tetris cognition and show that it cannot explain performance data from human players of the game-even when we relax the assumption of fully sequential processing. Standard models disregard many actions taken by players because they appear unmotivated or superfluous. However, we show that such actions are actually far from superfluous; they play a valuable role in improving human performance. We argue that traditional accounts are limited because they regard action as having a single function: to change the world. By recognizing a second function of action—an epistemic function—we can explain many of the actions that a traditional model cannot. Although our argument is supported by numerous examples specifically from Tetris, we outline how the new category of epistemic action can be incorporated into theories of action more generally.
Conference Paper
To be successful, ubicomp applications must be designed with their environment and users in mind and evaluated to confirm that they do not disrupt the users' natural workflow. Well-established techniques for understanding us- ers and their environment exist, but are not specifically designed to assess how well the computing and physical task environments blend. We present strengths and weaknesses of several qualitative and quantitative user study techniques for ubicomp. We applied these techniques to the design and evaluation of a ubi- comp application for cell biology laboratories (Labscape). We describe how these techniques helped identify design considerations that were crucial for Lab- scape's adoption and demonstrate their ability to measure how effectively appli- cations blend into an environment.
Conference Paper
To develop a comprehensive and systematic approach to the automated design of visual discourse, we introduce a visual task taxonomy that interfaces high-level presentation intents with low-level visual techniques. In our approach, visual tasks describe presentation intents through their visual accomplishments, and suggest desired visual techniques through their visual implications. Therefore, we can characterize visual tasks by their visual accomplishments and implications. Through this characterization, visual tasks can guide the visual discourse synthesis process by specifying what presentation intents can be achieved and how to achieve them.
Conference Paper
We argue that a comprehensive model of graph comprehension must include spatial cognition. We propose that current models of graph comprehen- sion have not needed to incorporate spatial processes, because most of the task/graph combinations used in the psychology laboratory are very simple and can be addressed using perceptual processes. However, data from our own re- search in complex domains that use complex graphs shows extensive use of spatial processing. We propose an extension to current models of graph com- prehension in which spatial processing occurs a) when information is not ex- plicitly represented in the graph and b) when simple perceptual processes are inadequate to extract that implicit information. We apply this model extension to some previously published research on graph comprehension from different labs, and find that it is able to account for the results.
Conference Paper
Visualization exploration is an iterative process of setting parame- ters, rendering, and evaluating results. This process can be recorded and analyzed in order to make visualization exploration more effi- cient and more effective. This work describes methods for visu- alizing the visualization process using new visualization process graphs; several visualization process relations are introduced to construct these graphs. These graphs and relations were used to an- alyze and improve a network routing visualization, and the results of this analysis are presented. Through this analysis, redundant ex- ploration was quickly identified and eliminated.
Article
Insight provenance – a historical record of the process and rationale by which an insight is derived – is an essential requirement in many visual analytics applications. Although work in this area has relied on either manually recorded provenance (for example, user notes) or automatically recorded event-based insight provenance (for example, clicks, drags and key-presses), both approaches have fundamental limitations. Our aim is to develop a new approach that combines the benefits of both approaches while avoiding their deficiencies. Toward this goal, we characterize users' visual analytic activity at multiple levels of granularity. Moreover, we identify a critical level of abstraction, Actions, that can be used to represent visual analytic activity with a set of general but semantically meaningful behavior types. In turn, the action types can be used as the semantic building blocks for insight provenance. We present a catalog of common actions identified through observations of several different visual analytic systems. In addition, we define a taxonomy to categorize actions into three major classes based on their semantic intent. The concept of actions has been integrated into our lab's prototype visual analytic system, HARVEST, as the basis for its insight provenance capabilities.
Article
Usability evaluation is an increasingly important part of the user interface design process. However, usability evaluation can be expensive in terms of time and human resources, and automation is therefore a promising way to augment existing approaches. This article presents an extensive survey of usability evaluation methods, organized according to a new taxonomy that emphasizes the role of automation. The survey analyzes existing techniques, identifies which aspects of usability evaluation automation are likely to be of use in future research, and suggests new ways to expand existing approaches to better support usability evaluation. Categories and Subject Descriptors: H.1.2 [Information Systems]: User/Machine Systems - human factors; human information processing; H.5.2 [Information Systems]: User Interfaces - benchmarking; evaluation/methodology; graphical user interfaces (GUI) Human Factors.
Article
Analysis and interpretation of behavior-environment relations are increasingly being conducted with data that have been derived descriptively. This paper provides an overview of the logic that underlies a sequential analytic approach to the analysis of descriptive data. Several methods for quantifying sequential relations are reviewed along with their strengths and weaknesses. Data from descriptive analyses are used to illustrate key points. Issues germane to contingency analysis in natural environments are discussed briefly. It is concluded that the conceptual distinctions among contiguity, contingency, and dependency are critical if the logic of sequential analysis is to be extended successfully to a behavior-analytic account of reinforcement in natural environments.