Content uploaded by Lynda Thomas
Author content
All content in this area was uploaded by Lynda Thomas
Content may be subject to copyright.
A preview of the PDF is not available
Debugging: The good, the bad, and the quirky -a qualitative analysis of novices' strategies
Abstract and Figures
A qualitative analysis of debugging strategies of novice Java programmers is presented. The study involved 21 CS2 students from seven universities in the U.S. and U.K. Subjects "warmed up" by coding a solution to a typical introductory problem. This was followed by an exercise debugging a syntactically correct version with logic errors. Many novices found and fixed bugs using strategies such as tracing, commenting out code, diagnostic print statements and methodical testing. Some competently used online resources and debuggers. Students also used pattern matching to detect errors in code that "just didn't look right". However, some used few strategies, applied them ineffectively, or engaged in other unproductive behaviors. This led to poor performance, frustration for some, and occasionally the introduction of new bugs. Pedagogical implications and suggestions for future research are discussed.
Figures - uploaded by Lynda Thomas
Author content
All figure content in this area was uploaded by Lynda Thomas
Content may be subject to copyright.
... The landscape of debugging education is characterized by diverse challenges and the pursuit of tailored pedagogical strategies. Novices often struggle with the daunting task of interpreting error messages and implementing effective fixes, frequently leading to frustration and wasted time on seemingly trivial issues [13]. Recognizing the recurring patterns in students' mistakes, educators are increasingly seeking to refine their curricula, thereby enriching the learning experience and equipping students with robust debugging skills [14]. ...
... Studies have explored novice debugging strategies, revealing that novices use a mix of effective and ineffective methods [13,27]. Effective strategies include rereading problem specifications and reexamining output, while ineffective ones involve random code changes. ...
... Understanding debugging through process data is critical. Think-aloud protocols and human observation provide detailed insights but are labor-intensive [7,13]. Automated collection of programming traces offers a scalable alternative, facilitating the extraction of performance indicators such as success rate and debugging time [6,21]. ...
In the field of software development, debugging is a crucial yet challenging skill, particularly for novice programmers. This study evaluates the efficacy of systematic debugging education in enhancing students’ debugging skills through two experimental phases. The first experiment involved 31 CS II students tackling debugging problems without prior instruction. The second experiment involved a different group of 59 students who received structured debugging instruction before engaging with similar debugging problems. The research aims to address the differences in debugging performance before and after the introduction of systematic debugging strategies, and the specific challenges faced by novice programmers in identifying and fixing various types of bugs. The results indicate significant improvements post-instruction, with the median correctness level rising from 60% to 80% and the median time to complete tasks decreasing from 28.7 minutes to 10.7 minutes. Further analysis revealed that certain bugs, such as division by integers and misusing switch statements, posed greater challenges initially but showed marked improvement post-instruction. This study underscores the transformative impact of systematic debugging education on novice programmers, enhancing both their accuracy and efficiency in debugging tasks.
... Según Murphy et al. (2008), la metacognición desempeña un papel fundamental en las actividades de programación, especialmente en los procesos de depuración y evaluación de estrategias de solución. Yadav et al., (2022) sostiene que los componentes del pensamiento computacional, en especial la abstracción, pensamiento algorítmico, depuración, descomposición, guardan íntima asociación con los procesos metacognitivos. ...
... Todas estas actividades se ven involucradas al momento de resolver un problema computacionalmente, cuando el sujeto repite un proceso buscando la solución más óptima, está iterando, cuando un sujeto revisa su propia ejecución, identifica y corrige errores, está depurando, cuando es consciente de la necesidad de descomponer un problema en partes que le sean más manejables, está descomponiendo. Esta ha sido la motivación para varios autores muestran especial interés en la relación entre estos dos procesos cognitivos las habilidades de pensamiento computacional por lo tanto guardan alta relación con procesos de monitoreo y conocimiento sobre sus propios procesos cognitivos (Murphy et al., 2008;Yadav et al., 2022). ...
Resumen: Las habilidades de pensamiento computacional, en especial la abstracción, pensamiento algorítmico, depuración, descomposición, guardan íntima asociación con los procesos metacognitivos. Dada la superposición entre ambos procesos, podría sugerir que la enseñanza del pensamiento computacional tiene el potencial desarrollar las estrategias metacognitivas de los estudiantes e influir en la resolución de problemas de los estudiantes y los resultados del aprendizaje (Yadav et al., 2022). El estudio actual tuvo el propósito de explorar el efecto de la enseñanza del pensamiento computacional sobre algunas habilidades metacognitivas como la memoria de trabajo. Consistió en un estudio cuasi experimental con diseño de pretest-postest con grupo control. Participaron 73 niños (grupo experimental N= 43; grupo control N= 30) de quinto grado de primaria. Como medida de resultado se utilizó el test de metamemoria de la batería BANFE-2 (Flores-Lázaro et al., 2014). La intervención consistió en el entrenamiento en pensamiento computacional COGNI-MACHINE, diseñado por la primera autora, el cual se desarrolló a modo de talleres dos veces por semana durante doce semanas. Los resultados del ANOVA-MR (Análisis de Varianza de Medidas Repetida)indicó que el grupo experimental, comparado con el grupo control, mostró una diferencia significativa para los indicadores de metamemoria total y errores positivos con tamaño del efecto mediano y pequeño respectivamente, luego del entrenamiento. Estos hallazgos sugieren que el aprendizaje del pensamiento computacional tiene un efecto favorable en el desarrollo de la metamemoria de los estudiantes.
... Our findings suggested a need for additional practice and time, modeling, and explicit teaching of CT concepts. For example, pattern recognition was a challenge for debugging found in several studies (Caci et al., 2013;McCauley et al., 2008;Murphy et al., 2008), especially for novices. In the present study, repeating one action was easier for novices to understand than using a nested loop. ...
Debugging is a growing topic in K-12 computer science (CS) education research. Although some previous studies have examined debugging behaviors, only a few have focused on an in-depth analysis of elementary students’ debugging behaviors in block-based programming environments. This qualitative study explored the debugging behaviors of four students, including their strategies and challenges. The study employed thematic video analysis of students’ computer screens as they engaged in block-based programming activities. The findings reveal five types of debugging strategies and three primary challenges during the debugging process. This study aims to help researchers and educators understand elementary students’ debugging strategies and the challenges they face. Suggestions for teaching debugging strategies to elementary students and the implications for future research are discussed.
... Good problem-solvers tend to be intentional with their behaviors in the game and use previous actions to influence their next move, which often leads to productive code edits (Liu & Israel, 2022;Liu & Moon, 2021). There is also qualitative evidence of unproductive code edits in Murphy et al. (2008), where novice computer programmers in college demonstrated a behavior called "tinkering." Students made "random and unnecessary changes" to their code that did not get them closer to the correct solution. ...
Computational thinking (CT) in young children (K to three) has been of much interest among educational researchers due to the applicability of CT to solving problems in daily life and various academic disciplines. This study uses existing data from children’s gameplay in a block-based programming game called codeSpark Academy to examine the extent to which we can use children’s gameplay behavior to measure their CT and, more generally, their problem-solving skills. The objectives of the study are to operationalize CT and problem-solving constructs using gameplay data, investigate the relationship between CT and problem-solving, and position codeSpark Academy as a valid assessment tool. A total of 72 elementary students (aged 6–9) played codeSpark Academy once a week for six weeks. TechCheck, an externally developed and validated measure of CT, was administered before the first game day and after the last game day. Using fine-grained, moment-to-moment gameplay data, we developed and validated seven game-based indicators (GBIs) of CT using correlational analysis and nonparametric tests and integrated them into a problem-solving framework. Our findings showed that children’s gameplay behavior can be used to measure their CT and problem-solving skills.
... In addition, it is also stated that it is more difficult to correct errors in a program than to detect them and that it becomes easier or harder for students to detect errors according to their experiences (Fitzgerald et. al, 2008;Lewis, 2012;Murphy et al., 2008). According to another study (Gugerty & Olson, 1986), students with no experience have relative difficulty in detecting errors. ...
The study aims to investigate prospective middle school mathematics teachers' mental actions related to debugging, which is one of the computational thinking skills in the modeling process. The study was conducted with a single-case embedded model. The collaborative working group consisted of three prospective mathematics teachers selected by criterion sampling. The data were collected from the video analysis, screen excerpts, and GeoGebra files explaining the solution process of three prospective mathematics teachers for the designed two mathematical modeling problems (experimental and theoretical). According to the results obtained from the data through content analysis based on the theoretical framework, it was identified that the prospective teachers conducted sub-activities such as recognizing/detecting the error, extracting the error, and correcting the error, which is one of the dimensions of computational thinking in technology-supported mathematical modeling. These skills as the basic steps of interpretation, verification, and revision were developed in the process of technology-supported mathematical modeling. GeoGebra was involved as an important mental trigger in the debugging process. In further studies, computational thinking studies describing all the components in the process of technology-supported mathematical modeling can be conducted, and computational thinking skills can be revealed in the process of mathematical modeling in non-computerized environments.
... I mostly just used trial and error to see what worked best.'' It is not unusual to see trial-and-error (e.g., Fitzgerald et al., 2010;Grigoreanu et al., 2006;Murphy et al., 2008) especially among novice programing learners. This is often the case in the blockbased programing contexts (Kim et al., 2018(Kim et al., , 2022. ...
In this qualitative study, preservice early childhood education teachers created block-based code to control robots and used the robots in field experience at local preschools. The study is grounded in a conceptual framework that weaves together playful programing and resilience, interlocking concepts that can explain sustained engagement during times of challenge. We investigated how and why preservice early childhood teachers exhibit resilience during collaborative programing. We analyzed their debugging processes, reflections, and interviews using a phenomenological lens. We conducted open and axial coding and analysis of discourse and actions during debugging episodes. Results suggest that teachers exhibited resilience due to the following three reasons: through playful coding, preservice early childhood teachers (a) learned that computer science is approachable and fun, (b) engaged in adaptive attribution, and (c) engaged in joint celebration when they observed each other’s successes during collaborative tinkering. These findings provide potential insights for teacher learning of computing but also for novices learning to program.
This article presents a series of four experiments investigating students' debugging of LISP programs. The experiments involve a population of students who know LISP reasonably well in that their errors are best classified as slips (Brown & Van Lehn, 1980). That is, students are unlikely to repeat the same errors either within their program or across programs (Experiment 1). The students' understanding of LISP is also reflected in their debugging behavior: They can usually fix a bug once they locate it. Students' difficulties are in locating the erroneous line of code. We observe that students use a variety of bug-location strategies during debugging (Experiment 2) and that the choice of strategy differs depending on whether students are debugging their own programs or other students' programs (Experiment 3). In addition, we observe that although the different bug-location strategies affect which lines of a program are searched, once students decide on a line, their ability to judge whether or not the line is correct and their ability to correct an error are not substantially affected by the strategy used to locate the line (Experiment 4). Finally, we argue that our results have implications not only for debugging in other computer languages, but for the general processes involved in troubleshooting as well.
The process by which students learn to program is a major issue in computer science educational research. Programming is a fundamental part of the computer science curriculum, but one which is often problematic. It seems to be difficult to find an effective method of teaching that is suitable for all students. In this research we tried to gain insights into ways of improving our teaching by a careful examination of students' mistakes. The compiler errors that were generated by their programs together with the pattern that was observed in their debugging activities formed the basis of this research. We discovered that many students with a good understanding of programming do not acquire the skills to debug programs effectively, and this is a major impediment to their producing working code of any complexity. Skill at debugging seems to increase a programmer's confidence and we suggest that more emphasis be placed on debugging skills in the teaching of programming.
Debugging is often difficult and frustrating for novices. Yet because students typically debug outside the classroom and often in isolation, instructors rarely have the opportunity to closely observe students while they debug. This paper describes the details of an exploratory study of the debugging skills and behaviors of contemporary novice Java programmers. Based on a modified replication of Katz and Anderson's study of novices, we sought to broadly survey the modern landscape of novice debugging abilities. As such, this study reports general quantitative results and fills in the picture with qualitative detail from a relatively small, but varied sample. Comprehensive interviews involving both a programming and a debugging task, followed by a semi-structured interview and a questionnaire, were conducted with 21 CS2 students at seven colleges and universities. While many subjects successfully debugged a representative set of typical CS1 bugs, there was a great deal of variation in their success at the programming and debugging tasks. Most of the students who were good debuggers were good novice programmers, although not all of the good programmers were successful at debugging. Students employed a variety of strategies to find 70% of all bugs and of the bugs they found they were able to fix 97% of them. They had the most difficulty with malformed statements, such as arithmetic errors and incorrect loop conditions. Our results confirm many findings from previous studies (some quite old) - most notably that once students find bugs, they can fix them. However, the results also suggest that some changes have occurred in the student population, particularly an increased use of debugging tools and online resources, as well as the use of pattern matching, which has not previously been reported.
Two experiments investigated expert-novice differences in debugging computer programs. Debugging was done on programs provided to the subject, and were run on a microcomputer. The programs were in LOGO in Exp. 1 and Pascal in Exp. 2. Experts debugged more quickly and accurately, largely because they generated high quality hypotheses on the basis of less study of the code than novices. Further, novices frequently added bugs to the program during the course of trying to find the original one. At least for these simple programs, experts superior debugging performance seemed to be due primarily to their superior ability to comprehend the program.
This paper reports the results of an exploratory study that investigated expert and novice debugging processes with the aim of contributing to a general theory of programming expertise. The method used was verbal protocol analysis. Data was collected from 16 programmers employed by the same organization. First, an expert-novice classification of subjects was derived from information based on subjects' problem solving processes: the criterion of expertise was the subjects' ability to chunk effectively the program they were required to debug. Then, significant differences in subjects' approaches to debugging were used to characterize programmers' debugging strategies. Comparisons of these strategies with the expert-novice classification showed programmer expertise based on chunking ability to be strongly related to debugging strategy. The following strategic propositions were identified for further testing. 1. (a) Experts use breadth-first approaches to debugging and, at the same time, adopt a system view of the problem area; (b) Experts are proficient at chunking programs and hence display smooth-flowing approaches to debugging. 2. (a) Novices use breadth-first approaches to debugging but are deficient in their ability to think in system terms; (b) Novices use depth-first approaches to debugging; (c) Novices are less proficient at chunking programs and hence display erratic approaches to debugging.
Ten experienced programmers were each given the same 12 FORTRAN listings to debug. Each listing contained a non-syntactic error in one line. Median debug times (7 min), number of bugs not found (11% of the listings), and number of incorrect assertions about the location of the bug (less than one per listing) all replicated earlier results (Gould & Drongowski, 1974). Although subjects were given the opportunity to use the interactive debugging facilities of an on-line computer, they rarely did so. Bugs in assignment statements were about three times as hard to detect as array or interaction bugs. Debugging was about three times as efficient on programs subjects had debugged previously (although with a different bug). A number of basic concepts relating to debugging are identified and a gross theory of debugging is described.
Students often "get stuck" when trying to learn new computing concepts and skills. In this paper, we present and categorize strategies that successful students found helpful in getting unstuck. We found that the students reported using a broad range of strategies, and that these strategies fall into a number of recognizably different categories.