No file available
To read the file of this research,
you can request a copy directly from the authors.
Attention in Software Maintenance: An Eye Tracking Study
Abstract and Figures
In software projects, people often continue working on programs that others created. This is especially the case in agile development. Therefore, developers often need to understand unfamiliar code in order to expand, change, fix or review it. Navigating through code and searching for relevant information requires a lot of developer time. This paper introduces an approach to use eye tracking to record and transfer developers' attention during software maintenance. We visualize the recorded data with heatmaps and display switches between Java classes by coloring filenames. These attention representations aim to help developers finding and navigating to relevant code sections faster.
We conducted an eye tracking experiment to investigate whether these attention visualizations help developers during software maintenance. The results show that both representations helped some of the participants for orientation and code finding purposes. The heatmaps slightly decreased the cognitive load as well. However, most participants, especially the more experienced ones, rated the attention visualizations as barely helpful or not helpful. Instead, the representations increased their visual effort, reduced the code's clarity and made the overall understanding of the program's functionality, as well as finding relevant areas, more difficult. By testing this specific implementation of the approach of attention transfer, we gained important lessons learned and provide several improvement suggestions.
Figures - uploaded by Maike Ahrens
Author content
All figure content in this area was uploaded by Maike Ahrens
Content may be subject to copyright.
ResearchGate has not been able to resolve any citations for this publication.
Eye-movement modeling examples (EMME) have previously been shown to impact attention of learners in various domains. Potentials of such training methods, however, have remained unexplored in computer programming education. In the current study, participants attended to a classroom experiment in which their comprehension strategies were cued by an educational intervention based on theories of program comprehension (Block Model) and accompanied by a visualization of eye-movement strategies of an advanced programmer. EMME-based interventions affected gaze following slightly, and resulted in improved problem-solving. We present results related to the gaze-behavior during the intervention and demonstrate how participants overlapped their attention with the expert. The findings inform about the design of eye-movement modeling examples in computing education.
Information Foraging Theory (IFT) has established itself as an important theory to explain how people seek information, but most work has focused more on the theory itself than on how best to apply it. In this paper, we investigate how to apply a reactive variant of IFT (Reactive IFT) to design IFT-based tools, with a special focus on such tools for ill-structured problems. Toward this end, we designed and implemented a variety of recommender algorithms to empirically investigate how to help people with the ill-structured problem of finding where to look for information while debugging source code. We varied the algorithms based on scent type supported (words alone vs. words + code structure), and based on use of foraging momentum to estimate rapidity of foragers' goal changes. Our empirical results showed that (1) using both words and code structure significantly improved the ability of the algorithms to recommend where software developers should look for information; (2) participants used recommendations to discover new places in the code and also as shortcuts to navigate to known places; and (3) low-momentum recommendations were significantly more useful than high-momentum recommendations, suggesting rapid and numerous goal changes in this type of setting. Overall, our contributions include two new recommendation algorithms, empirical evidence about when and why participants found IFT-based recommendations useful, and implications for the design of tools based on Reactive IFT.
Even when working on a well-modularized software system, programmers tend to spend more time navigating the code than working with it. This phenomenon arises because it is impossible to modularize the code for all tasks that occur over the lifetime of a system. We describe the use of a degree-of-interest (DOI) model to capture the task context of program elements scattered across a code base. The Mylar tool that we built encodes the DOI of program elements by monitoring the programmer's activity, and displays the encoded DOI model in views of Java and AspectJ programs. We also present the results of a preliminary diary study in which professional programmers used Mylar for their daily work on enterprise-scale Java systems.
Large software projects often require a programmer to make changes to unfamiliar source code. This paper presents the results of a formative observational study of seven professional programmers who use a conventional development environment to update an unfamiliar implementation of a commonly known video game. We describe several usability problems they experience, including keeping oriented in the program's source text, maintaining the number and layout of open text documents and relying heavily on textual search for navigation. To reduce the cost of transferring knowledge about the program among developers, we propose the idea of wear-based filtering, a combination of computational wear and social filtering. The development environment collects interaction information, as with computational wear, and uses that information to direct the attention of subsequent users, as with social filtering. We present sketches of new visualizations that use wear-based filtering and demonstrate the feasibility of our approach with data drawn from our study.
Large software projects often require a programmer to make changes to unfamiliar source code. This paper describes a set of tools, called Team Tracks, designed to ease program comprehension by showing the source code navigation patterns of fellow development team members. One technique shows a list of related items, given that the user is viewing a given method or class. Another technique shows the favorite classes, by showing a class hierarchy view that hides less frequently visited classes, methods, and members. Two user studies, a laboratory study and a field study, were run to evaluate the effectiveness of these techniques. The results of the two studies demonstrate that sharing navigation data can improve program comprehension and is subjectively preferred by users.
When working on a large software system, a programmer typically spends an inordinate amount of time sifting through thousands of artifacts to find just the subset of information needed to complete an assigned task. All too often, before completing the task the programmer must switch to working on a different task. These task switches waste time as the programmer must repeatedly find and identify the information relevant to the task-at-hand. In this paper, we present a mechanism that captures, models, and persists the elements and relations relevant to a task. We show how our task context model reduces information overload and focuses a programmer's work by filtering and ranking the information presented by the development environment. A task context is created by monitoring a programmer's activity and extracting the structural relationships of program artifacts. Operations on task contexts integrate with development environment features, such as structure display, search, and change management. We have validated our approach with a longitudinal field study of Mylar, our implementation of task context for the Eclipse development environment. We report a statistically significant improvement in the productivity of 16 industry programmers who voluntarily used Mylar for their daily work.
Much of software developers' time is spent understanding unfamiliar code. To better understand how developers gain this understanding and how software development environments might be involved, a study was performed in which developers were given an unfamiliar program and asked to work on two debugging tasks and three enhancement tasks for 70 minutes. The study found that developers interleaved three activities. They began by searching for relevant code both manually and using search tools; however, they based their searches on limited and misrepresentative cues in the code, environment, and executing program, often leading to failed searches. When developers found relevant code, they followed its incoming and outgoing dependencies, often returning to it and navigating its other dependencies; while doing so, however, Eclipse's navigational tools caused significant overhead. Developers collected code and other information that they believed would be necessary to edit, duplicate, or otherwise refer to later by encoding it in the interactive state of Eclipse's package explorer, file tabs, and scroll bars. However, developers lost track of relevant code as these interfaces were used for other tasks, and developers were forced to find it again. These issues caused developers to spend, on average, 35 percent of their time performing the mechanics of navigation within and between source files. These observations suggest a new model of program understanding grounded in theories of information foraging and suggest ideas for tools that help developers seek, relate, and collect information in a more effective and explicit manner
In this paper we present a work in progress proposal to create a dynamic help system to help learners solving programming tasks. This help system uses social navigation and guidance based on asynchronous gaze sharing. The navigation and guidance part depend on traces of eye-tracking data of peer learners created during their problem-solving. The goal is to create an interactive learning experience based on behavioral data of peer learners that supports new learners while learning to program. The help system is integrated in a self-developed programming environment to use the full potential of components like code editor, API documentation, task description and error messages. As a result, this environment will be a personal learning environment regarding the behavioral data of peer learners.
Computer-supported collaboration changed the way we learn and work together, as co-location is no longer a necessity. While presence, pointing and actions belong to the established inventory of awareness functionality which aims to inform about peer activities, visual attention as a beneficial cue for successful collaboration does not. Several studies have shown that providing real-time gaze cues is advantageous, as it enables more efficient referencing by reducing deictic expressions and fosters joint attention by facilitating shared gaze. But the actual use is held back due to its inherent limitations: Real-time gaze display is often considered distracting, which is caused by its constant movement and an overall low signal-to-noise ratio. As a result, the transient nature makes it difficult to associate with a dynamic stimulus over time. While it is helpful when referencing or shared gaze is crucial, the application in common collaborative environments with a constant alternation between close and loose collaboration presents challenges. My dissertation work will explore a novelty gaze sharing approach, that aims to detect task-related gaze patterns which are displayed in concise representations. This work will contribute to our understanding of coordination in collaborative environments and propose algorithms and design recommendations for gaze sharing.
Remote collaboration can be more difficult than collocated collaboration for a number of reasons, including the inability to easily determine what your collaborator is looking at. This impedes a pair's ability to efficiently communicate about on-screen locations and makes synchronous coordination difficult. We designed a novel gaze visualization for remote pair programmers which shows where in the code their partner is currently looking, and changes color when they are looking at the same thing. Our design is unobtrusive, and transparently depicts the imprecision inherent in eye tracking technology. We evaluated our design with an experiment in which pair programmers worked remotely on code refactoring tasks. Our results show that with the visualization, pairs spent a greater proportion of their time concurrently looking at the same code locations. Pairs communicated using a larger ratio of implicit to explicit references, and were faster and more successful at responding to those references.
[Context and motivation] Writing good specifications is difficult and takes time. There are several guidelines such as the Volere template to assist writing a good specification. They provide a table of contents which can be used like a checklist to consider all relevant aspects. Voluminous specifications take more time to write, and also more time to read. A larger specification is not always a better one. [Question/Problem] A requirements engineer should be aware of how readers make use of a specification and consider their interests in writing it. In addition, some people prefer reading on a screen while others hold a preference for paper printouts. Some parts or aspects may be read differently in both representations. [Principal ideas/results]: We have conducted an Eye Tracking study investigating how specifications are read. We compared paper-based with on-screen presentation, and different reading perspectives such as UI designers, tester, software architects etc. We derived study goals by using GQM down to the level of quantitative and statistical eye tracking analyses. [Contribution]: There is a two-fold contribution: (a) Observations and findings about the way specifications are read; e.g., we had expected paper-based reading to be faster. Instead, we found similar reading patterns on paper versus on screen. (b) Insights with respect to eye tracking as a research method for requirements engineering. We discuss strengths and shortcomings, and provide lessons learned.
The more we know about software developers’ detailed navigation behavior for change tasks, the better we are able to provide effective tool support. Currently, most empirical studies on developers performing change tasks are, however, limited to very small code snippets or limited by the granularity and detail of the data collected on developer’s navigation behavior. In our research, we extend this work by combining user interaction monitoring to gather interaction context-the code elements a developer selects and edits -with eye-tracking to gather more detailed and fine-granular gaze context-code elements a developer looked at. In a study with 12 professional and 10 student developers we gathered interaction and gaze contexts from participants working on three change tasks of an open source system. Based on an analysis of the data we found, amongst other results, that gaze context captures different aspects than interaction context and that developers only read small portions of code elements. We further explore the potential of the more detailed and fine-granular data by examining the use of the captured change task context to predict perceived task difficulty and to provide better and more fine-grained navigation recommendations. We discuss our findings and their implications for better tool support.
The experiment data from the operation is input to the analysis and interpretation. After collecting experimental data in the operation phase, we want to be able to draw conclusions based on this data. To be able to draw valid conclusions, we must interpret the experiment data.
As with any engineering discipline, software development requires a measurement mechanism for feedback and evaluation. Measurement is a mechanism for creating a corporate memory and an aid in answering a variety of questions associated with the enactment of any software process. It helps support project planning (e. g., How much will a new project cost?); it allows us to determine the strengths and weaknesses of the current processes and products (e. g., What is the frequency of certain types of errors?); it provides a rationale for adopting/refining techniques (e. g., What is the impact of the technique XX on the productivity of the projects?); it allows us to evaluate the quality of specific processes and products (e. g., What is the defect density in a specific system after deployment?). Measurement also helps, during the course of a project, to assess its progress, to take corrective action based on this assessment, and to evaluate the impact of such action.
Before performing a modification task, a developer usually has to investigate the source code of a system to understand how to carry out the task. Discovering the code relevant to a change task is costly because it is an inherently human activity whose success depends on a large number of unpredictable factors, such as intuition and luck. Although studies have shown that effective developers tend to explore a program by following structural dependencies, no methodology is available to guide their navigation through the typically hundreds of dependency paths found in a non-trivial program. In this paper, we propose a technique to automatically propose and rank program elements that are potentially interesting to a developer investigating source code. Our technique is based on an analysis of the topology of structural dependencies in a program. It takes as input a set of program elements of interest to a developer and produces a fuzzy set describing other elements of potential interest. Empirical evaluation of our technique indicates that it can help developers quickly select program elements worthy of investigation while avoiding less interesting ones.
Expertise in computer programming can often be difficult to transfer verbally. Moreover, technical training and communication occur more and more between people who are located at a distance. We tested the hypothesis that seeing one person's visual focus of attention (represented as an eyegaze cursor) while debugging software (displayed as text on a screen) can be helpful to another person doing the same task. In an experiment, a group of professional programmers searched for bugs in small Java programs while wearing an unobtrusive head-mounted eye tracker. Later, a second set of programmers searched for bugs in the same programs. For half of the bugs, the second set of programmers first viewed a recording of an eyegaze cursor from one of the first programmers displayed over the (indistinct) screen of code, and for the other half they did not. The second set of programmers found the bugs more quickly after viewing the eye gaze of the first programmers, suggesting that another person's eye gaze, produced instrumentally (as opposed to intentionally, like pointing with a mouse), can be a useful cue in problem solving. This finding supports the potential of eye gaze as a valuable cue for collaborative interaction in a visuo-spatial task conducted at a distance.
In modern multi-representational environments, software developers need to coordinate various information sources to effectively perform maintenance tasks. Although visual attention is an important skill in software development, our current understanding of the role of visual attention in the coordination of representations during maintenance tasks is minimal. Therefore, we applied eye-tracking to capture visual attention strategies and construct a detailed account of visual attention during debugging. Two groups of programmers with two distinct levels of experience debugged a program with the help of multiple representations. The proportion of time spent looking at each representation, the frequency of switching attention between visualrepresentations and the type of switch were investigated during consecutive phases of debugging. We found repetitive patterns in visual strategies that were associated with less expertise and lower performance. Novice developers made use of both the code and graphical representations while frequently switching between them. More experienced participants expended more efforts integrating the available information and primarily concentrated on systematically relating the code to the output. Our results informed us about the differences in program debugging strategies from a fine-grain, temporal perspective and have implications for the design of future development environments.
Towards self-adaptive IDEs
- Roberto Minelli
Roberto Minelli. Towards self-adaptive IDEs. Proceedings -30th International Conference on Software Maintenance and Evolution, ICSME
2014, page 666, 2014.
Code Scanning Patterns in Program Comprehension
- Christoph Aschwanden
- Martha Crosby
Christoph Aschwanden and Martha Crosby. Code Scanning Patterns in
Program Comprehension. Proceedings of the 39th Hawaii International
Conference on System Sciences, 2006.
Rationale as a By-Product. Rationale Management in Software Engineering
- Kurt Schneider
Kurt Schneider. Rationale as a By-Product. Rationale Management in
Software Engineering, pages 91-109, 2006.