Spatial Reasoning and Debugging With Lego Structures

Abstract

Our study builds upon K-2 students' early block building experiences and the increased importance of fostering computational thinking skills, especially debugging. We had 12 K-2 students debug a Lego structure by finding and fixing differences between a given and target structure. Half of the students worked with red Lego bricks only while the other half worked with various colors of bricks. We analyzed the type and order of errors they fixed and their process for debugging. Misplaced Lego bricks on the plate were the most difficult for students to find and fix.

Full text

ICLS 2023 Proceedings © ISLS
1422
Spatial Reasoning and Debugging With Lego Structures
Sezai Kocabas, Yi Zhu, Laura Bofferding
Skocabas@purdue.edu, Zhu966@purdue.edu, Lbofferd@purdue.edu
Purdue University
Abstract: Our study builds upon K-2 students’ early block building experiences and the
increased importance of fostering computational thinking skills, especially debugging. We
had 12 K-2 students debug a Lego structure by finding and fixing differences between a given
and target structure. Half of the students worked with red Lego bricks only while the other
half worked with various colors of bricks. We analyzed the type and order of errors they fixed
and their process for debugging. Misplaced Lego bricks on the plate were the most difficult
for students to find and fix.
Introduction
With the increased focus on computational thinking, considering how students use computational thinking skills
(e.g., debugging) across non-programming domains such as block building is important (Barr & Stephenson,
2011). In block building, debugging can involve identifying and fixing differences between two structures
(Kocabas et al., 2022), a type of change detection task (Rensink et al., 1997; Shore et al., 2016). In recent research,
K-2 students built a vertical Lego structure. They were then told to find and fix differences between their
completed structure and a modified one using a building manual (Kocabas et al., 2022). The students successfully
engaged in debugging using strategies similar to those in programming. However, students who used a grayscale
manual (as opposed to color) identified fewer differences, and the symmetrical features of the vertical structure
played a role in their debugging (Kocabas et al., 2022). Students’ efforts to build the original structure may have
aided their efforts to debug it. In this study, we expand on prior findings by investigating students’ spatial thinking
as they debug a combination horizontal-and-vertical structure when they did not build the original structure. Our
research questions include the following: How do students debug a multi-colored versus a red Lego structure
when referring to pictures of the target structure and a video of the structure being composed? (a) Which bugs do
students fix and in what order? (b) What debugging strategies do students use?
Spatial reasoning and change-detection framework
Spatial visualization skill is “…generating and manipulating mental images of two- and three-dimensional objects,
including moving, matching, and combining them” (Clements & Sarama, 2009, p.110; see also intrinsic spatial
skills, Newcombe & Shipley, 2015). Children ages 5-8 (similar to our participants) develop from being able to
slide, flip, and turn shapes (sometimes in the wrong direction) to being able to use mental images to predict the
results of moving shapes (Clements & Sarama, 2009). Further, they develop from local framework users that can
interpret “objects’ positions relative to landmarks” (p. 118) to being able to interpret locations based on
coordinates (Clements & Sarama, 2009; see also extrinsic spatial skills, Newcombe & Shipley, 2015). If the
landmarks are not obvious, K-2 students might struggle to place blocks along the forward and backward dimension
in relation to those landmarks (e.g, Kocabas et al., 2022). Block building activities support these spatial reasoning
abilities (e.g., Brosnan, 1998; Casey et al., 2008), part-whole reasoning (Casey et al., 2008), and visual-difference
recognition.
Two types of visual change-detection tasks include identifying differences after sequentially seeing two
versions of a scene (e.g., Shore et al., 2006; Stieff et al., 2020) or seeing two versions of the scene side-by-side
(see “Hidden Pictures” tasks, https://www.highlightskids.com/games). Fewer elements in the scene or central
errors make differences easier to spot (Rensink et al., 1997; Shore et al., 2006). Additionally, while deleted objects
are the hardest to detect, changes in object position and orientation are easier to spot than changes in color
(Rensink et al., 1997; Shore et al., 2006). In contrast, visual changes such as color on digital molecular
representations were easier to detect than other changes such as spatial grouping for undergraduate students (Stieff
et al., 2020). Debugging is a critical process in programming and non-programming activities such as puzzles or
block building (Bofferding & Kocabas, 2021; Ahn, et al., 2021; Caeli & Yadav, 2020). However, in change
detection debugging tasks with a Lego structure, students were most likely to find errors at the top and bottom of
a vertical structure, along with differences in orientation and placement. Errors in the middle of the structure were
hardest to find (Kocabas et al., 2022). They also used spatial thinking techniques like visualization and rotation
and similar debugging strategies as in programming (Kocabas et al., 2022; Murphy et al., 2008, see Table 1).

ICLS 2023 Proceedings © ISLS
1423
Methods
Participants and setting
In this exploratory study, we recruited four kindergarteners, four first graders, and four second graders from an
afterschool program at a Montessori school in the midwestern United States. During each session, students met
with one of the researchers one-on-one in a classroom. As students worked with the Lego bricks, we used two
cameras to record their work from the front and back of the structures.
Design
The data comes from the third out of three sessions of a larger project. During this session, we gave students an
already-built Lego structure (randomly assigned to using multicolored or red bricks) and asked them whether they
would like to use the pictures or video to debug the six bugs in the Lego structure (see Figure 1A & 1B). They
could switch between these media whenever they wanted. As they debugged, we asked students questions about
where they were looking in the picture or video, how they identified erroneous bricks, and how they knew how
to fix errors. We also gave some hints (e.g., “A structure was placed in the wrong spot on this green plate.”) if the
students were stuck or feeling frustrated. We analyzed how students debugged the six bugs on the Lego structure
in the third session of the project. We first identified which bugs each student debugged and explored the sequence
of debugging among the bugs. We also identified if they introduced a new bug when debugging. Then, we coded
the students’ debugging process based on programming and block debugging strategies (see Table 1; Kocabas et
al., 2022; Murphy et al., 2008, see also Bofferding et al., 2022).
Table 1
Description of Program Debugging Strategies and Potential Application to Block Debugging Strategies
Debugging strategy
Programming description
Finding bugs in block structures
Tinkering
Students randomly making changes;
multiple changes at one time.
Students play with the blocks, randomly
adding or removing pieces to see the result.
Tracing (top-to-
bottom or bottom-to-
top)
Students follow each line of code
and match it with the result,
reasoning forward or backward.
Students start at the top or bottom of the
structure and match it sequentially with the
steps in building the structure.
Pattern matching
Students just know what is wrong
or recognize the problem based on
past experience.
Students notice a difference by looking at the
overall structures. They may move their eyes
back and forth to similar parts of the structure
and picture to notice what has changed.
Rebuilding
Students scrap the code and
reprogram from the beginning.
Students take apart the structure and rebuild it
using the target structure by using the manual.
Understanding the
Structure
Students reason about how the code
works and the problem.
Students turn the physical structure or take
apart parts to see how they are put together.
Results
Bugs and order
Overall, students who debugged the multicolor structure found roughly one more bug on average (M=5.25,
Mdn=5.75) than students in the red structure group (M=4.17, Mdn=5). All students found and fixed Bug5, which
involved rotating the eye, making it the easiest bug. In fact, three students in the multicolor group and four students
in the red group found it first. Students were less successful at fixing the rotation issue with Bug4 with only nine
students doing so (and it was often the fifth bug they found). Therefore, although students could rotate bricks
(Clements & Sarama, 2009), identifying the differences in orientation was easier when the brick had a distinct
feature. Students who did not find rotation Bug5 first often found the missing piece instead (Bug2) or fixed the
piece with an incorrect thickness (Bug1). Eleven students (six multicolor, five red group) found Bug2, followed
by nine students (five multicolor, four red group) who fixed Bug1. The difference between groups increased for
correcting Bug3 on the staircase; five students from the multicolor group fixed it, while three students from the

ICLS 2023 Proceedings © ISLS
1424
red group did so. Students in the multicolor group not only had to fix the block from a 2x3 to a 2x4, but they also
had to change the color. While these students noticed both parts of the bug (if they found the bug), one student
noticed the change in color later.
Even though the video showed Bug6a and Bug6b first, most students fixed these errors last (if at all),
and most students needed the hint that one of the structures was put in the wrong place. Once they figured out that
the tree was in the wrong place, five students from the multicolor group and four students from the red group
fixed Bug6a, which involved adjusting the tree structure outward (from three to two studs from the edge), farther
away from the staircase structure. Four students from the multicolor group and only two students from the red
group also fixed 6B by adjusting the structure inward (from four to six studs from the edge).
Debugging strategies
Groups used similar debugging strategies and referred to the video and pictures of the final structures to varying
extents. Two students only relied on the pictures to debug, six students started with the pictures and then switched
to the video to help them as needed, and four students started with the video and then switched to the pictures as
needed. No students only used the video to find the differences, but all of them turned the structure at some point
to understand the structure and check for differences. Only one first grader used a rebuilding strategy (see Figure
1C) when trying to correct differences in a chunk. She fixed Bug2 and Bug3, but then later removed the front arch
with tower and the staircase that led from it. She re-placed Bug1 (placing the thin piece back) but put it back in
the wrong place as she attempted to recreate the staircase. Further, she did not space the stairs correctly as she
rebuilt, so the placement of her fixed Bug3 ended up in the wrong place. She did not replace the front arch and
tower (including the fixed Bug2) before deciding to be done. All other students fixed bugs by removing the pieces
directly around the bug and placing them back immediately. As the students corrected the Lego structure, they
employed pattern matching and tracing strategies while moving between the pictures and the video. Additionally,
each student utilized the understanding of structure strategy by turning the structure to identify any differences.
Nonetheless, in some cases, specific students chose to look at the structure from different viewpoints instead of
turning it. When fixing Bug6a and Bug6b, students usually counted the studs on the base (Figure 1D), and only
one second grader counted the studs on blocks in her process to fix Bug3.
Figure 1
(A) Given Structure and Target Structure, (C) Rebuilding the structure, (D) Counting the studs on the base
(a)
(b)
(c)
(d)
Incorrect pieces: Bugs 1 and 3, Missing pieces: Bug 2, Incorrect orientation: Bugs 4 and 5,
Incorrect placement: Bugs 6a and 6b
Discussion and significance
Unlike in previous change detection tasks with Lego structures, we organized the structures horizontally.
Although Bug5 was shown third, students were most likely to fix it first, easily turning the eye. This result suggests
they could anticipate the rotation and that pictures on the bricks made them central to the Lego scene. It took
longer for students to notice orientation Bug4 compared to Bug5, further suggesting that students might easily
notice spatial bugs involving orientation when some visual feature makes it stand out more (Stieff et al., 2020).
Contrary to traditional change detection tasks where identifying missing elements are hardest (Rensink et al.,
1997), identifying the missing brick (Bug2) was one of the first two bugs students noticed, similar to prior Lego
debugging results (Kocabas et al., 2022), even for five students who were only looking at the images. Changes in
block size (Bug3 on the stairs) were harder for students in the red group to notice; students in the color group
might have found this bug easier to find because it was also in a different color. Future explorations should
determine if pairing bugs could make them more salient to students.
Interestingly, changes in placement (Bug6a and Bug6b) took the longest for students to find and were
hardest (with many needing hints). Therefore, placement bugs on the Lego plate appear to be more difficult to
detect than placement bugs on a structure (Kocabas et al., 2022). Students had an easier time detecting differences
in placement when the tree structure needed to be shifted away from another structure as opposed to shifted to the

ICLS 2023 Proceedings © ISLS
1425
right in relation to a structure, suggesting the saliency of proximity to landmarks (Clements & Sarama, 2009;
Newcombe & Shipley, 2015). In addition, students counted the studs on the base to identify and fix the placement
errors. Having more parts of the block design placed horizontally (as opposed to vertically), may have encouraged
some students to use mathematical concepts such as counting and helped move them toward seeing the Lego plate
as a form of a coordinate system made up of the studs. Future work could further explore factors that encourage
students’ use of the Lego plate as a coordinate system versus proximity to other landmarks when placing pieces.
For debugging, the findings were consistent with previous studies that block play can engage children in
debugging in early years of education. Compared to our previous analysis where students debugged after building
a structure (Kocabas et al., 2022), none of the students completely started over building from the beginning and
followed along with the video to build the corrected structure. In fact, only one student completely removed a
chunk, and that student had difficulty putting it all back together. Building a structure first may help students’
rebuilding; whereas fixing one structure at a time may have been easier given the several parts of this design.
References
Ahn, J., Sung, W., & Black, J. B. (2021). Unplugged debugging activities for developing young learners’
debugging skills. Journal of Research in Childhood Education, 36(3), 1-17.
https://doi.org/10.1080/02568543.2021.1981503
Barr, V., & Stephenson, C. (2011). Bringing computational thinking to K-12: What is involved and what is the
role of the computer science education community?. ACM Inroads, 2(1), 48-54.
Bofferding, L., & Kocabas, S. (2021). Elementary students’ use of spatial thinking strategies in a layered puzzle
task. In M. Inprasitha, N. Changsri, N. Boonsena (Eds.), Proceedings of the 44th Conference of the
International Group for the Psychology of Mathematics Education (Vol. 1, pp. 129). PME.
Bofferding, L., Kocabas, S., Aqazade, M., Haiduc, A., & Chen, L. (2022). The effect of play and worked examples
on first and third graders’ creating and debugging of programming algorithms. In A. Ottenbreit-Leftwich
& A. Yadav (Eds.), Computational thinking in PreK-5: Empirical evidence for integration and future
directions. A Special Research Publication (pp. 19-29). Association for Computing Machinery, Inc. and
the Robin Hood Learning + Technology Fund.
Brosnan, M. J. (1998). Spatial ability in children's play with Lego blocks. Perceptual and Motor Skills, 87(1), 19-
28.
Caeli, E. N., & Yadav, A. (2020). Unplugged approaches to computational thinking: A historical perspective.
TechTrends, 64(1), 29-36. https://doi.org/10.1007/s11528-019-00410-5
Casey, B. M., Andrews, N., Schindler, H., Kersh, J. E., Samper, A., & Copley, J. (2008). The development of
spatial skills through interventions involving block building activities. Cognition and Instruction, 26(3),
269-309. https://doi.org/10.1080/07370000802177177
Clements, D.H.; Sarama, J. (2009). Learning and teaching early math: The learning trajectories approach;
Routledge: London, UK.
Kocabas, S., Bofferding, L, Zhu Y., & Liang Y. (2022). The role of color in debugging Lego structures. In C.
Chinn, E. Tan, C. Chan, & Y. Kali, (Eds.). (2022). Proceedings of the 6th International Conference of
the Learning Sciences. (pp. 1449 – 1452). Hiroshima,Japan: International Society of the Learning
Sciences.
Murphy, L., Lewandowski, G., McCauley, R., Simon, B., Thomas, L, & Zander, C. (2008). Debugging: The good,
the bad, and the quirky – a qualitative analysis of novices’ strategies. SIGCSE ‘08 (pp. 163-167),
Portland, OR: ACM. https://doi.org/10.1145/1352322.1352191
Newman, S. D., Loughery, E., Ecklund, A., You, C., Von Werder, H., Soylu, F. (2020). Structured versus free
block play: The impact on arithmetic processing. Trends in Neuroscience and Education, 22, 100146.
Rensink, R.A., O’Regan, J.K., & Clark, J.J. (1997). To see or not to see: The need for attention to perceive changes
in scenes. Psychological Science, 8, 368–373.
Shore, D. I., Burack, J. A., Miller, D., Joseph, S., & Enns, J. T. (2006). The development of change detection.
Developmental Science, 9(5), 490-497. https://doi.org/10.1111/j.1467-7687.2006.00516.x
Stieff, M., Werner, S., DeSutter, D., Franconeri, S., & Hegarty, M. (2020). Visual chunking as a strategy for
spatial thinking in STEM. Cognitive Research: Principles and Implications, 5, 1-15.
https://doi.org/10.1186/s41235-020-00217-6
Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33-35.
Acknowledgements
This work was supported by a Ross-Lynn grant through Purdue University.