Teresa Busjahn’s research while affiliated with HTW Berlin - University of Applied Sciences and other places

The study of eye movements in programming has many implications to software engineering and computing education in industry and academia. The 10th International Workshop on Eye Movements in Programming (EMIP 2022) was co-located with the 44th International Conference on Software Engineering (ICSE), with the goal to further develop the methodology of using eye tracking for programming and software engineering, both theoretically and in practice.

A large dataset that contains the eye movements of N=216 programmers of different experience levels captured during two code comprehension tasks is presented. Data are grouped in terms of programming expertise (from none to high) and other demographic descriptors. Data were collected through an international collaborative effort that involved eleven research teams across eight countries on four continents. The same eye tracking apparatus and software was used for the data collection. The Eye Movements in Programming (EMIP) dataset is freely available for download. The varied metadata in the EMIP dataset provides fertile ground for the analysis of gaze behavior and may be used to make novel insights about code comprehension.

The methodology of eye tracking has been gradually making its way into various fields of science, assisted by the diminishing cost of the associated technology. In an international collaboration to open up the prospect of eye movement research for programming educators, we present a case study on program comprehension and preliminary analyses together with some useful tools. The main contributions of this paper are (1) an introduction to eye tracking to study programmers; (2) an approach that can help elucidate how novices learn to read and understand programs and to identify improvements to teaching and tools; (3) a consideration of data analysis methods and challenges, along with tools to address them; and (4) some larger computing education questions that can be addressed (or revisited) in the context of eye tracking.

While knowledge about reading behavior in natural-language text is abundant, little is known about the visual attention distribution when reading source code of computer programs. Yet, this knowledge is important for teaching programming skills as well as designing IDEs and programming languages. We conducted a study in which 15 programmers with various expertise read short source codes and recorded their eye movements. In order to study attention distribution on code elements, we introduced the following procedure: First we (pre)-processed the eye movement data using log-transformation. Taking into account the word lengths, we then analyzed the time spent on different lexical elements. It shows that most attention is oriented towards understanding of identifiers, operators, keywords and literals, relatively little reading time is spent on separators. We further inspected the attention on keywords and provide a description of the gaze on these primary building blocks for any formal language. The analysis indicates that approaches from research on natural-language text reading can be applied to source code as well, however not without review.

Programming is an intertwined process of reading and writing. So far, computing education research has often focused on the writing part. This paper takes a further look into the role of reading source code in learning to program. In order to complement the findings from literature, we conducted interviews with programming instructors using the miracle question, on the role of code reading and comprehension. The analysis of these interviews describes this role in terms of the five categories conceptualization, occurrences, and effects of successful code reading, challenges for learners, as well as approaches to facilitate code reading. As a result, we suggest to take a further look into the different reading processes involved in programming, in order to add to the knowledge about programming instruction.

Code reading, although an integral part of program comprehension, is rarely reflected. In this paper, we want to argue for a research approach and direction exploiting the potential that lies in the analysis of reading processes. Based on the vast experience compiled in psychology and some studies in computing, eye tracking and think aloud were elaborated as a viable research instrument for code reading studies. We conducted a feasibility study, designed to examine the actual process of code reading as the sensory starting point of comprehension. Computational and statistical tools were developed to facilitate data capture and analysis for eye tracking experiments. Results do not just provide proof of concept but already emphasize differences between reading natural language text and source code, as well as a distinct attention allocation within different code elements like keywords and operators. In conclusion we suggest a combination of theory-driven selected stimuli material, a carefully designed procedure of eye tracking, complemented with suitable post-tests on comprehension as well as retrospective think aloud in order to obtain additional information on the linking process between perception and comprehension. As an addition to other research approaches this should most certainly help us to improve our knowledge of comprehension within an educational research framework.