ArticlePDF Available

Concepts before coding: non-programming interactives to advance learning of introductory programming concepts in middle school


Abstract and Figures

Background and Context: Learners struggle with conceptual understanding of introductory programming concepts such as variables, expressions, and loops. Objective: We examine whether and how designed activities for conceptual exploration support preliminary engagement with and learning of foundational and often hard-to-grasp programming concepts for students in grades 6–8. Method: Drawing on principles from dynamic mathematics, we developed a suite of non-programming digital and unplugged activities embedded in a curriculum before students engage in Scratch block-based programming. We conducted empirical research in three middle school classrooms in diverse urban US schools and examined student performance through mixed qualitative and quantitative methods. Findings: Learning gains were significant and not predicted by grade, gender or prior academic preparation. Free-choice projects of students showed statistically greater (correct) use of key concepts compared to those not in the study. Implications: Our work demonstrates the promise of novel approaches such as interactive non-programming activities for deeper understanding of programming concepts.
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
Computer Science Education
ISSN: 0899-3408 (Print) 1744-5175 (Online) Journal homepage:
Concepts before coding: non-programming
interactives to advance learning of introductory
programming concepts in middle school
Shuchi Grover, Nicholas Jackiw & Patrik Lundh
To cite this article: Shuchi Grover, Nicholas Jackiw & Patrik Lundh (2019): Concepts before
coding: non-programming interactives to advance learning of introductory programming concepts in
middle school, Computer Science Education, DOI: 10.1080/08993408.2019.1568955
To link to this article:
Published online: 06 Feb 2019.
Submit your article to this journal
Article views: 13
View Crossmark data
Concepts before coding: non-programming interactives
to advance learning of introductory programming
concepts in middle school
Shuchi Grover
, Nicholas Jackiw
and Patrik Lundh
Looking Glass Ventures, Palo Alto, CA, USA;
SRI International, Menlo Park, CA, USA
Background and Context: Learners struggle with concep-
tual understanding of introductory programming concepts
such as variables, expressions, and loops.
Objective: We examine whether and how designed activities
for conceptual exploration support preliminary engagement
with and learning of foundational and often hard-to-grasp
programming concepts for students in grades 68.
Method: Drawing on principles from dynamic mathematics,
we developed a suite of non-programming digital and
unplugged activities embedded in a curriculum before stu-
dents engage in Scratch block-based programming. We
conducted empirical research in three middle school class-
rooms in diverse urban US schools and examined student
performance through mixed qualitative and quantitative
Findings: Learning gains were signicant and not predicted
by grade, gender or prior academic preparation. Free-choice
projects of students showed statistically greater (correct) use
of key concepts compared to those not in the study.
Implications: Our work demonstrates the promise of novel
approaches such as interactive non-programming activities
for deeper understanding of programming concepts.
Received 5 August 2018
Accepted 9 January 2019
Computer science
education; CS concepts;
introducing programming;
novice programmer;
curriculum design; K-12 CS
education; variables;
expressions; Boolean logic;
Policy and educational leaders see computer science (CS) skills as necessary for
all citizens, not just computer scientists, in order to build a strong STEM and
computing pipeline and to develop future citizens with the problem-solving
abilities needed to thrive and innovate in a world driven by computing and
digital devices (The White House, 2016). Learning to program is a key ingre-
dient of introductory CS curricula in K-12 classrooms. Learners today are
typically introduced to programming in block-based programming environ-
ments designed to provide engaging features and syntactical supports for
novice learners (Bau et al., 2017). However, programming is a complex activity
CONTACT Shuchi Grover Chief Learning Scientist, Looking Glass Ventures,
202 Sequoia Avenue, Palo Alto, CA 94306, USA
© 2019 Informa UK Limited, trading as Taylor & Francis Group
that novices of all ages nd dicult regardless of the introductory program-
ming environment (e.g. Robins, Rountree, & Rountree, 2003). Weak mental
models and conceptual understandings prevent deeper learning of founda-
tional CS concepts (Mayer, 2004; Robins et al., 2003). Franklin et al. (2017) and
Grover, Pea, and Cooper (2015,2016) report that students struggle more with
certain introductory concepts than others in the context of programming. As
middle school students learn to reason abstractly in the Piagetian formal
operational stage, they need pedagogically robust experiences that provide
a foundation for key concepts in CS and programming.
To address the need to expand teaching and learning of programming and
CS, we explore new pedagogical approaches that support deeper conceptual
understanding. Our research explores the impact of interactive, non-
programming activities and technology-based microworlds on middle school
studentsconceptions of variables, expressions, loops and abstraction. We see
arich and dynamic understanding of variables as a cornerstone of students
encounters with all of these VELAconcepts: variables (V) are the edice of
expressions (E); which in turn are critical linguistic drivers of iterative and
looping (L) control structures; and variables are seen in CS education as the
gateway to broader understandings of computational abstraction (A) and
generalization (Figure 1). Recognizing the correlations between mathematics
and programming understanding (as seen in Grover et al., 2015; Lewis & Shah,
2012; Shute, 1991), we draw inspiration from mathematics education research,
where a pioneering curricular technology the dynamic mathematics repre-
sentation, has demonstrated positive eects in conceptual mathematics learn-
ing. For example, research on the use of SimCalc (an interactive mathematics
software that employs a dynamic representational approach) in three large-
scale studies with students in diverse demographic settings, established strong
benets to student conceptual understanding when dynamic representations
are integrated into the plan for a curricular lesson sequence, and teachers have
received related teacher professional development (Roschelle et al., 2010).
Figure 1. How abstraction builds on variables, expressions, and loops.
This paper rst briey describes ve of the six activities of our VELA project:
Story Variables, Cats and Ladders, Graphical Looping, Three Switches, and Alarm
Clock that were designed, piloted and rened through a design-based research
process (Wang & Hannan, 2005). We then describe the empirical research
conducted to examine the eectiveness of our intervention within an intro-
ductory CS course in three public middle school classrooms in a diverse urban
school district in the US. We end with a discussion of our ndings and plans for
future work.
Literature review and theoretical framework guiding pedagogical
Our focus on the VELA concept trajectory is based on the large body of
research devoted both to the diculties of learning programming generally
(for example, Du Boulay, 1986; Lahtinen, Ala-Mutka, & Järvinen, 2005; Pea &
Kurland, 1984; Robins et al., 2003), and to studentsspecicdiculties with our
focal concepts (for example, misconceptions around variables, or pervasive
diculties with variables in loops and abstraction). Today, block-based pro-
gramming environments such as Scratch, Alice, or Snap! designed to
address syntactical diculties, are typical vehicles for introducing novice lear-
ners in K-12 to computational thinking and programming. Yet research shows
that even though students nd these visual environments easier to navigate,
their conceptual diculties pertaining to the semantics of coding still persist
(Franklin et al., 2017; Grover & Basu, 2017; Grover et al., 2015).
Research documents diculties with the concept of a variable in introduc-
tory programmer experiences, especially around initialization and misinterpre-
tations of the common boxor drawerdidactic metaphor for storage (Du
Boulay, 1986; Samurcay, 1989). Novices struggle with naïve notions about
variable assignment (assuming variables can have multiple values at the
same time); distinguishing between what goes inside a loop and what pre-
cedes or follows a loop (Du Boulay, 1986); and designing mathematical and
logical expressions and naming variables (Gobil, Shukor, & Mohtar, 2009).
While providing tools for inspecting variables as the program executes has
been helpful in debugging code as well as understanding control and program
state (Pears et al., 2007), such tools do not always help learners grasp how
variables, expressions, and loops mutually interact to solve (or understand)
a complex problem. Grover et al. (2015) found that seventh- and eighth-
graders who had typical value-in-a-box variable visualization tools available
(as in Scratch) still had trouble conceptualizing variables and how to use them
in code, especially in constructing terminating conditions for loops involving
variables and Boolean expressions (Grover et al., 2015). This echoes earlier
research that problems pertaining to the understanding of conditionals are
attributed to a lack of understanding of Boolean operators (Ebrahimi, 1994).
More recently, Grover and Basu (2017) also found that some students believe
a variable must be a letter that is used as a short form for an unknown number
(a carry-over from algebraic treatments of variable in math class), and students
often assume multiple commands in a loop are repeated not as a unit but
rather individually one after the other.
Past research also faults the possible lack of more pedagogic attention to
and engagement with conceptual ideas for the problems novices face on
their road to a robust understanding of programming (Pea & Kurland, 1984).
Most K-12 programming classrooms are committed to student-centered activ-
ity and hands-on engagement with programming. But these activities are
often centered on open-ended programming with minimal guidance, resulting
in a lack of deeper or explicit engagement with, and understanding of,
programming concepts and patterns (Mayer, 2004).
Pedagogical approach#1: constructivist engagement with concepts before
Our pedagogic approach in designing student-centered conceptually focused
microworlds is inuenced by constructivism with its twin emphases on knowl-
edge as constructed (as opposed to transmitted) and on material engagement
and manipulation as eective activity in such knowledge production
(Ackerman, 2001).
We believe conceptually focused, constructivist experiences oer class-
rooms a path out of the minimally guided approaches critiqued above,
through designed opportunities for students to acquaint themselves with
conceptual knowledge through intentional student activity that also accommo-
dates exploration. Our digital microworlds thus focus on concepts rather than
constructs. This contrasts with much of the innovation that has occurred in the
design of programming environments themselves (like Scratch, App Inventor,
Blockly, PencilCode, Snap!, etc.), where students actually code and run pro-
grams. We design non-programming learning activities that allow students to
explore and construct ideas for themselves, while also encountering powerful
ideas embedded within the activities, thus balancing exploration and gui-
dance before they employ them in code (in a programming environment).
This also draws on learning theory emphasizing that encountering concepts in
multiple contexts helps learners to abstract the relevant features of concepts
and hence promote deeper conceptual understanding (Bransford, Brown, &
Cocking, 2000). Also, constructivist approaches to learning point to providing
learners opportunities to actively construct knowledge by using their intuition
and prior knowledge and rening their micro-theories as they interact with
artifacts before being provided explanations (Schwartz & Bransford, 1998).
Additionally, since contextualizing computational concepts makes it easier to
learn programming (Papert, 1991), all our activity designs (as well as
accompanying ideas for discussions and examples) are situated in relatable
real-world contexts. We also strive to explicitly bridge non-programming activ-
ities and subsequent introductory programming activities in Scratch for med-
iating transfer between the two contexts (Grover et al., 2014).
Pedagogical approach#2: design principles drawn from dynamic geometry
Our technology environment design draws on the innovations of dynamic
geometry software in mathematics education over the past 30 years, as typied
by The Geometers Sketchpad (Jackiw, 19912009) and Cabri Géometre (Laborde
& Strässer, 1990)two tools that have had global impact on mathematics
instruction. Considerable research has explored the potential of dynamic
geometry software in mathematical and curricular areas beyond its traditional
home in high-school geometry (e.g. Schattschneider & King, 1997), often
through connections predicated upon geometrys traditional status as the
home of mathematical insightand upon dynamisms particularly eective
relationship to mathematical ideas of the real continuum and the variable. We
see dynamic representations as a potentially powerful vehicle for conceptual
engagement among diverse learners in CS classrooms given its success in
mathematics classrooms in large urban school districts (Roschelle et al., 2010).
From our synthesis of dynamic geometrysoftware(acrossaspectrumofmath-
ematical applications and variety of age levels), we developed design principles for
the construction of introductory VELA activities and microworlds. These include the
principles of: (1) ubiquitous consistency,wherebyatanyinstantintime,asystemof
relationships is always entirely self-consistent: components of that system always
accurately reect their fundamental denitions or relationships to other compo-
nents of the system, and there is no uninitialized value;(2)dynamic variation,
whereby time is used as the principle axis of interrogation for both graphical and
numerical parameters; varying these parameters oers tremendous potential for
insight into the behavior of the system across inputs; (3) graphical presentation,
whereby diagrams and visualizations are treated as primary representations and
manipulatives in knowledge production, thus emphasizing the insightful value of
the illustrative dimension of images and diagrams, as well as evolve them through
interactivity into engines of knowledge production rather than just presentation;
and, nally (4) incremental abstraction,wheretheserst three principles contribute
to learning trajectories in which abstraction is achieved incrementally through
structured temporal variation. We believe these principles oer promise as an
approach to balancing the tensions betweenanoverly-materialhands-onprogram-
concepts) and an overly theoretical or foundational approach to essential deni-
tions(that misses the chance for experiential learning from the behavior of objects
and systems). For more details on these principles and their manifestations in VELA
activity designs, see Grover, Jackiw, and Lundh (n.d.).
VELA activities design
In this section, we describe ve of the VELA activities (four digital and one
unplugged) that we conceptualized and iteratively rened as part of the larger
suite of both digital and non-digital activities for use in introductory CS class-
rooms to support early exploration and understanding of the VELA concepts in
middle school (Table 1).
Wiggins & Tighe (2005)s Understanding by Design framework guided our
activity design process that began with an articulation of the precise target
learning goals for our activities and companion curriculum (Table 2). Our
digital activities were developed using a combination of Web Sketchpad and
Javascript, and were deployed on the web ( Over a 15-
month period, our multi-disciplinary research team comprising CS education
and math education researchers, learning scientists, and a software developer
iteratively designed and rened the activities based on inputs from teachers
and students. Initial activity concepts were designed with classroom teachers
from a large, urban school district in Western USA using a participatory design
Although each activity is intended to be modular and standalone so that it
can be embedded in any introductory programming curriculum, we suggest
a sequence. Our activities on looping and variables require no prior frame of
reference, but activities on expressions draw on prior work with variables.
Abstraction, as the capstone of our curricular materials, can be seen to draw
on both looping activitiesintroduction of execution ow and blocksor
chunksof functionality, as well as on the variable-and-expression activities
introduction of derived and compound data types. In the former context, we
focus on control abstraction and in the latter on data abstraction.
Designed as a short and preliminary exploration and introduction to key ideas of
foundational concepts (rather than introductory programming constructs) these
activities do not attempt to deliver comprehensive treatments and are designed
for one (or two) class periods. All digital activities begin with students exploring
the basic phenomenology of the microworld. They are designed for exploration
among student pairs interspersed with whole-class discussions.
Specically, VELA activities introduce
Table 1. Suite of VELA activities [gray row indicates activity not discussed in this paper].
Name Type Introductory Computing Concept(s)
Graphical Looping Digital Repeating pattern, Looping, Modeling
Story Variables (lead-in to Cats and
Unplugged Variables (basic idea of variation, naming, datatypes)
Cats and Ladders Digital Variables, Arithmetic Expressions
Dice Conditionals Unplugged Conditionals, Boolean outcomes (TRUE/FALSE)
Three Toggles (VEA Microworld
Digital Boolean Logic, Boolean Expressions
Alarm Clock (VEA Microworld
Digital Arithmetic and Boolean Expressions, Data Types, String
Data, Modeling
Variables as named quantities that can change value
Expressions as dening new changing values by applying value-
appropriate operations to existing variables
Looping as repetitions of some identiable repeating unit
Abstraction as the process of giving a name to a specic collection of
details as a way of referencing its purpose without quoting or enumerat-
ing its detail.
We describe the activities and lesson sequence in brief in the remainder of this
Variables and arithmetic expressions: Story Variables (unplugged) and Cats
and Ladders (digital)
The Cats and Ladders activity engages students with the concept of variables. It
follows an unplugged (o-line) activity, Story Variables.InStory Variables
students work collaboratively in pairs to investigate a series of short stories
containing quantities that vary (Figures 2 and 3)). For example, Excuse me
last week I bought one of these pens here for $1.50. Are you really telling me
they now cost $3?;I watched the basketball game last night. At halftime we
were tied, but in the end, they beat us 9490. Through discussions, students
Table 2. VELA learning goals.
Students will learn how programs are executed sequentially
Students will learn how simple loops work (xed, predetermined number of repetitions)
Students will learn how to create dierent pathways in programs using conditional statements
Students will learn algorithmic ow of controlhow instructions are executed in sequence even when there
are loops, except that the set of instructions within a loop are repeated
Students will understand that in control structures (like loops and conditionals), a collection of an arbitrary
number of statements can be declared to act as a single statement by grouping them
Students will understand what Initialization (in general and of variables, specically) is and why it is needed
Students will understand that variables can only hold one value at a time
Students will learn how types dene the set of values a variable can have, and the set of operators that can
be used
Students will learn how variables are created, used, assigned values and updated
Students will learn how variable values change within loops
Student will learn what initialization is and why it is important
Students will learn how to use expressions to make new variables from old ones
Boolean variables, operators and expressions
Students will learn the idea of controlling loops and conditionals using Boolean conditions (that may or may
not involve variables and expressions)
Boolean as a data type
Students will practice identifying and articulating patterns in real-world phenomena and problems, and
abstracting them into structural components of a program (preconditions, repeating logic in a loop, any
Students will learn how variables are an abstraction or representation of data in the program and the real
Students will learn the importance of planning before programming
Students will learn the need for breaking down problems into smaller manageable tasks
Students will learn computational solutions are abstractions; and that these abstractions can be represented
in dierent ways
come up with a denition of variable,practice identifying and naming
variables meaningfully, and analyze a variables changing values to determine
its specic types and expected ranges. Students articulate their own real-world
scenarios that involve variables. As a nal activity, they watch a video clip of
a video game (e.g. Pacman) and list the dierent variables they observe, their
values in the course of the video clip, and the logical ranges of values for each.
Cats and Ladders formalizes this understanding of variables as named quantities
with specicvaluesthatcanchange.IntheCats and Ladders digital activity (Figure 4),
Figure 2. Short everyday narratives in story variables.
Figure 3. Identifying variables, meaningful naming, and values in story variables.
students rescue distraught cats from the various oors of buildings by determining
the length of the ladder required to reach them. The activity is divided into stages.
Starting with an exploration of the basic behavior of the microworld, new stages are
unlockedas the student proceeds through theactivities.Studentsmovefrom
working with one (blue) ladder, to two stacked ladders (one orange and one blue)
when the blue ladder height is insucient to rescue cats on higher oors. Learners
discuss appropriate names for variables (e.g. heightor LadderHeightis not
suciently discriminating for the two ladders). They also discuss the range of
possible values (often determined by context), and that dierent variables may
naturally reference dierent (data) types. Finally, they engage in abstraction through
synthesized from existing ones (e.g. TotalLadderHeight =BlueLadderHeight +
The gamesnal stage includes multiple buildings with alphabetic labels
(Figure 4(d)) aimed at helping students interact with a two-dimensional coor-
dinate system. In comparing this third variable whose values are letters,
rather than numbers to the other two, students encounter an early example
of non-numeric and compound data types. From here they discuss possible
further generalizations of the scenario.
Looping (repetition): Graphical Looping (digital)
The Graphical Looping activity sequence (Figure 5) introduces students to
iterated repetitions of a block of actions within a sequence of events, and
develops the idea that we eciently express such a ow of events in terms of
a more compact specication of that repetition. Students engage with the idea
of action sequences that occur before and/or after a repeating chunk of
actions (repeating unit). Graphical Looping uses comic panels as a proxy for
Figure 4. Screenshots of cats and ladders activity.
source code in a pre-programming context. Comic strips are atextual (and thus
do not disadvantage ELL students) and contain a formal, and block-structured,
grammar for describing action sequences familiar to students.
First, in Swimming Pool Story Arrangement, students arrange comic panels to
tell a logical story. Then, they proceed From Arranging to Generating think-
ing about how panels to the story could describe longer swims and identify
the inner storyor repeating unit. Students answer questions such as: What
happens when the inner story is repeated zero times in the results? Once? If
the repeat count is 10, how many pool lengths will the swimmer swim? How
many panels will be in the result? How might the total length swum increase
and the swimmers energy decrease with each lap? These ideas are later
revisited as examples in bridging VELA activities to Scratch.
Boolean operators and expressions, and abstraction: Three Switches
(digital) and Alarm Clock (digital)
The last two digital activities in the VELA sequence introduce learners to
Boolean operators and expressions, and, nally, abstraction through the con-
crete trope of naming a complex phenomenon in order to hide its details and
complexity behind a simple name. Although distinct activities, both Three
Switches and Alarm Clock are built on the same underlying expressionauthor-
ing microworld. In keeping with the VELA design philosophy, both activities
are situated in real-world scenarios.
In Three Switches (Figure 6), the goal is twofold: (1) to explore the basic mechanics
of the expression-authoring features of the microworld, and (2) to provide learners
with a rst introduction to Boolean values, operators, and expressions. Students
explore how Boolean operators (AND, OR, NOT) work, and what the resulting
expressions evaluate to using an example of three switches that toggle between
Figure 5. Screenshots of graphical looping activity.
two states (ON and OFF/TRUE and FALSE). They start with a series of four explora-
tory challengesthat explain the types of Boolean operations. In Challenge #1,
students discover through exploration that the bulb is controlled only by switch
A. When A is on, the bulb is on.InChallenge#2Studentsnd out that the bulb
turns on when either Switch A is on, or Switch C is on, or both switches A and C are
on. (Switch B does not control the bulb). In Challenge #3, the bulb turns on only
when both switches B and C are on; and in Challenge #4, when Switch A is on, the
bulb is o;andwhenSwitchAiso,thebulbison.Thenextstepoftheactivity
reveals to students that the expressions that control the bulb in challenges 14are:
A, A OR C, B AND C, and NOT A, respectively. Students then ll out truth tables for
expressions A OR C, B AND C, and NOT A, and explain the behavior of the OR, AND,
and NOT operators in words.
Students subsequently extend the ideas to create and evaluate Boolean
expressions that use these operators either as simple expressions or
Figure 6. Screenshots of three switches activity.
compound expressions that combine one or more operators. The activity ends
with an exploration of connections between variables and Boolean expressions
in real-world scenarios through discussions that involve articulating the out-
come variable, the controlling variables, the Boolean operators used, and the
Boolean expression. For example
(1) Travisparents say that he can go out and play with his friends if he has
done both nished his homework and made his bed in the morning
(2) Maya will go on a hike if it is not raining.
(3) Keisha can watch a movie if she nishes either her math homework or
her computer science project.
(4) A car gives a warning beep if
Drivers seat-belt is not locked
Passengers seat-belt is not locked
Car is being driven
In the culminating VELA activity, Alarm Clock (Figure 7), students construct a family
of arithmetic and logical expressions using a variety of data types and starting from
should ring to wake them for school (when IsAlarmTime is True). Beginning,
again, with phenomenologicalinvestigation,workinAlarm Clock is conducted in
an expression-building microworld, in which a dynamic simulation agraphical
alarm clock may be probed (analyzed, measured, mathematically modeled) by
interrogating the values of several key variables, and by combining them with
various operators into dynamic expressions. By assigning names to these new
expressions, and by incorporating those new names (rather than the full symbolic
expressions dening them) into new expressions that reference those intermediate
results, students develop a hierarchy of abstractions to manage the burgeoning
symbolic and conceptual complexity of a family of equations expressed only in
terms of foundational variables.
Through this iterative revisiting, the modeling context is open to a variety of
denitions of success depending on the interests and ability of the students. Some
go on to aspire to only be woken for schools on weekdays,andextendtheir
expressions further. In such a model, an expression such as (isTimeForSchool AND
isSchoolDay) becomes a highly abstract proposition, encoding (and hiding) a non-
trivial amount of decision-making involving times, clock mechanics, and day-
Empirical study: methods, data measures and analyses
Organizing the VELA curricular intervention
Working with classroom teachers, the VELA activities were incorporated into
a20-day intervention (Table 3) to be embedded within an introductory pro-
gramming middle school curriculum. The VELA intervention comprised the
VELA activities
These included six VELA activities (the ve described above and Dice
Conditionals, an unplugged dice game activity to introduce students to true/
false logic and conditional thinking, described in Grover, Lundh, & Jackiw,
2019), along with accompanying teacher lesson plans, student worksheets
(to be completed in pairs in conjunction with the activities), and review
sheetsfor formative assessment.
Scratch lessons
In addition to the VELA activities, we designed lesson plans to introduce students
to basic Scratch constructs and programming concepts. The Scratch lesson plans
also made connections to relevant ideas in VELA activities in an eort to mediate
transfer. For example, the Scratch activities related to loops with variables made
explicit connections to Graphical Looping. Learners are introduced to the idea of
Figure 7. Screenshots of alarm clock activity.
using a Repeatblock to swim three laps, change the (swimmers) Energy value,
and also watchthis value decrease (using the Sayblock) with each lap (or
iteration of the loop). They then progressed to the idea of a generalized solution
where the number of laps was based on an input from the user to be used as
a variable in the Repeat block (Figure 8).
The bulk of Scratch programming involved hands-on work in Scratch
completing partially coded programs using the relevant construct being intro-
duced, creating complete programs to achieve a goal, and/or open-ended
programs of choice. We also devoted attention to the construction and use
of Boolean expressions and their use in conditional statements and the
Repeat Untilloop block a non-deterministic loop construct that makes
the loop repeat until some criterion is met. It requires the construction of
a Boolean expression to determine when it ends, and the expression could use
a variable that gets updated in the loop. This is conceptually more sophisti-
cated than the simpler (and more commonly taught) form of looping taught in
Table 3. 20-day VELA intervention to be embedded in an introductory CS class.
VELA activities Descriptions
Day-0 Pre-assessment and Pre-survey
Day-1 Graphical Looping (Digital Activity)
Swimming comic strip and other real
world examples
Identifying repeating patterns and conceptualizing
before/during/after of loops
Day-2 Simple Loops (Scratch) Scaolded and open-ended Scratch activities using
simple Repeat, Drawing in Scratch, basic shapes.
Day-3 Nested Loops Flowers and music (Scratch) Scaolded and open-ended Scratch activities using
nested Repeats
Day-4 Simple Conditionals (Scratch) Scratch activities (sensing and conditional in Maze
Day-5 Story Variables (Unplugged Activity) Introducing idea of changing quantities in the real
world and giving them meaningful names
Day-6 Cats and Ladders (Digital Activity) Naming variables and creating expressions in non-
programming context.
Day-7 Creating and Updating Variables (Scratch) Creating variables in Scratch; understanding how to
initialize and update values
Day-8 Unplugged Dicegame Arithmetic epressions and conditionals based on
Day-9 Conditionals with Arithmetic Expressions
Scaolded and open-ended Scratch activities using
variables, expressions and conditionals.
Day-10 Loops with Varialbles (Scratch) Scaolded and open-ended Scratch activities using
variables, and loops
Day-11 Loops, Variables and Conditionals (Scratch) Scaolded and open-ended Scratch activities using
variables, loops and conditionals
Day-12 VEA 3 Switches (Digital Modelling Activities) Booleans, boolean operators and abstractions
Day-13 Games with Booleans (Scratch) Scaolded and open-ended Scratch activities using
boolean logic
Day-14 VEA Alarm Clock (Digital Modelling
Modelling real world Alarm Clock situtions using
variables, arithmetic and logical expressions and
Day-15 Games using Repeat-Unitls (Scratch) Scaolded and open-ended Scratch activities using
Repeat-Unitls loops, variables, expressions and
Games in Scratch Make games more exciting by using all concepts
Day-21 Post-assessment and post-survey
Scratch –“Foreverloops with an IF-block conditional check within them that
is at odds with accepted practice of CS (Meerbaum-Salant, Armoni, & Ben-Ari,
2011). As with VELA activities, worksheets guided students work during Scratch
Formative assessments
Every VELA activity and Scratch programming lesson included review questions to
be completed by students individually. These extended studentsthinking beyond
the examples encountered in the lesson, provided feedback to teachers on student
understanding, and informed subsequent whole-class reviews and discussions.
Pre-post assessment
We also designed, piloted and developed an assessment instrument using
Evidence-Centered Design (Mislevy, Almond, & Lukas, 2003), a principled
assessment design framework, to be used as a pre-post measure. The targeted
skills were drawn from VELA learning goals as well as the K-12 CS framework
(, and CSTA CS standards (2017, draft version at the time) so that the
instrument could be usable more broadly for assessing introductory programming
skills in middle school CS. The assessment was rened based on data from pilots
in middle school classrooms teaching introductory Scratch programming
(~200 students over 2 pilots), teacher feedback, student cognitive think-
alouds and language simplication by an English Language Learner (ELL)
expert. The 9 items, with 29 individually scorable sub-items, were
a combination of multiple-choice and open response types; and used Scratch
code snippets or non-programming, real-world scenarios. One sample ques-
tion is shown in the Appendix. The entire assessment, along with diculty,
reliability and validity analyses, is described in Grover (2019).
(a) Rewrite this code
snippet using a simple
loop. The swimmer starts
with an energy of 1500
calories. You should “say”
how many calories are left
at each iteration of the
(b) Now, you do not know beforehand how many laps the swimmer will swim.
Open the starter program. You will see code (below) that asks the user to enter how many
laps the swimmer will swim (a number between 1 and 10).
Complete the program so that based on the number the user enters, the program will
show how the swimmer’s energy decreases. Show how many calories are left at each
iteration of the loop.
Figure 8. Example of programming activities in the scratch that bridge to VELA.
Classroom research
We partnered with a large urban school district in Western USA for this
research. IRB permissions were sought from teachers and students (and par-
ents of students) involved in this research.
Beta versions of the VELA activities were piloted in controlled settings at
three dierent time points before they were used in a classroom research
study (1) within our research lab during early prototyping, (2) at a local
summer camp in Northern California in 11 and small group sessions with six
middle school children and (3) with 16 students with varied levels of prior
math preparation from two dierent middle schools in our partner district.
Data from these pilots helped rene the design of the activities before the full
classroom intervention.
The VELA curricular intervention was implemented in the partner district by
three middle school CS teachers in sixth, seventh and eighth grades (each in
adierent school) with a total of 74 students. (Three students were dropped
from data analyses due to missing permissions or pre-posttest data for a nal
N= 71 students). While the seventh grade was in a high-performing school,
the other two schools were low performing. Grade 7 also had statistically
higher prior math and ELA scores than Grades 6 and 8. Table 45describes
the sample by gender, grade and ethnicity, and the school descriptions.
Before implementing the VELA curriculum, the teachers participated in two
days of professional development. Teachers learned about the VELA rationale
and the principles guiding the curriculum design. They participated in role-
playing during curriculum enactment, engaged in hands-on activities to
experience using the curriculum and reviewed lesson plans. They were encour-
aged to modify any lesson plan as well as tweak the pace and timeline of the
curriculum, if necessary.
We conducted mixed-method (including quasi-experimental) research to
determine (a) whether and how well students learned through the VELA curri-
cular intervention, and (b) how student learning was linked to studentsdemo-
graphic factors (such as prior math and English preparation, gender, ethnicity and
grade level). These research questions were also aimed at understanding the
Table 4. Student sample and demographics.
Grade 6 Grade 7 Grade 8
Male 17 15 11
Female 9 16 3
Asian 17 22 7
American Indian 1 0 0
White 1 2 0
Declined to State 0 5 0
Multiracial 2 1 1
Filipino 0 1 3
Hispanic/Latino 4 0 3
African American 2 0 0
design modication needs of the VELA activities as guided by DBRs iterative
approach. A third, broader thread of inquiry not covered in this paper involves
conducting and analyzing case studies of the classroom activity system (including
student and teacher cases) in order to address the question: What factors and
conditions should be highlighted in a theory of action for engaging diverse
middle school CS classrooms, including factors related to students, to teachers,
to resources and to the environment?
Data sources and analysis
We collected qualitative and quantitative data to inform our analyses of
student learning and experiences with VELA activities. Data measures included:
Pre-post assessment: a paperpencil assessment completed in one class
Student Interviews: of four students from each class conducted twice
during and after the intervention.
Individual Teacher Interviews and Focus Group sessions: with the three
teachers piloting the VELA curricular intervention (for us to receive feed-
back on the VELA activities)
Post-VELA Surveys: with Likert-scale items addressing engagement with
VELA activities and understanding of VELA concepts.
Prior Math and English Scores: from the standardized state tests were
provided by the district for each student in our sample.
Final, free-choice (open-ended) projects in Scratch (VELA Students): from the
three classrooms, done (individually or in pairs) as a culminating activity of
the CS course in which the VELA intervention was embedded. As this was
completed at the end of the 9-week CS course (well after the end of the
VELA intervention), we received a total of only 54 projects across the three
Final, free-choice (open-ended) projects in Scratch (non-VELA Students):as
a comparison, we also examined 60 nal Scratch projects provided by 20
teachers from middle school classrooms across the same district that were
not part of this study. These students completed the middle school CS
Table 5. School (grade) characteristics.
School 1 (Grade 6) School 2 (Grade 7) School 3 (Grade 8)
Low SES 85% 40% 87%
Math Prociency 55% 79% 34%
English Prociency 54% 80% 41%
Science Prociency 62% 84% 34%
Disabilities 12.8% 3.5% 15%
Facility Condition Good Exemplary Good
course adapted from Harvey MuddsMyCS curriculum (Schoeld, Erlinger,
& Dodds, 2014) as part of the districts ongoing CS expansion eorts.
These projects were randomly selected by teachers and similarly done by
individuals or pairs of students as a culminating activity.
This paper reports on analyses of data from the pre-posttest, post-VELA
surveys and interviews, and mixed-methods analyses of studentsopen-
ended Scratch projects. We conducted paired t-tests of pre- and post-test
scores, statistical comparisons and correlational analyses (Pearson correlations,
regressions) of posttest scores and learning gain (learning gain = posttest
pretest score) by gender, grade, and prior math and English scores.
Scoring of open-ended projects
We scored studentsopen-ended free-choice Scratch projects using an exten-
sive rubric from Grover, Basu, and Schank (2018) publicly available at https:// The rubric, adapted from Grover (2014), was created mainly for
evaluating Scratch projects rather than providing students a project score
based on a traditional student-facing rubric. It also included assessments of
program complexity, creativity and correctness. Figure 9 shows the main
dimensions of the Scratch rubric –“general factors,mechanics of design,
user experience,basic coding constructs, and advanced coding con-
structs, and what each of them measures. The separation of user experience
and design mechanicsis useful as it separates design mechanics (design
pattern-like code) such as using random motion, collision detection, and
keyboard input from the elements of creativity students use in Scratch as
part of engagement and user experience. General Factorsassess overall
prociency such as novelty, correctness (or bugginess), and complexity.
Advanced Coding Constructsincludes the types of constructs that suggest
more sophisticated program structures, for example, Repeat-Untilloops, Wait
Figure 9. Rubric dimensions for analysis of scratch projects, and examples of criteria within
each rubric dimension.
Until, methods, and cloning. These two category scores are weighted higher
than other categories in the overall score as they are reective of higher
program sophistication (and by extension, deeper conceptual understanding).
Additionally, since frequency of a construct is not necessarily indicative of
program sophistication, the rubric-based coding/scoring was based on one
of the following: (1) determinations about the existence or not (1/0) of a given
criterion (e.g. Does the program terminate at some point? Are instructions
provided for interactive elements? Do variables have meaningful names?), (2)
count of the number of times a construct was correctly used (most were
capped at 10 to account since code segments in Scratch are often repetitive),
(3) score along a 3-point scale (e.g. many bugs, few bugs, no bugs). In projects
that included several sprites with identical or near-identical code, code asso-
ciated with only one sprite was considered.
We ran two-sample t-tests to compare these scores and sub-scores to those
in the comparison dataset (both sets were scored using the same rubric) along
the rubric dimensions. We also specically compared the use (frequency and
correctness) of variables, Boolean and relational operators and various types of
loops (especially Repeat-Untilloops). We analyzed the post-VELA surveys as
well as student and teacher interviews for quantitative and narrative feedback
on the VELA activities.
Empirical results and discussion
We report on the quantitative analyses related to the pre-post assessment and
qualitative scoring of nal Scratch projects.
Pre-to-post scores
This section presents salient ndings from statistical analyses of posttest scores
and learning gain.
All three classrooms showed statistically signicant gains on average on the
pre-post assessment (See Table 6). Grade 7 students had a statistically higher
average pre-score than Grade 6 (t(55) = 5.83); p< 0.001). Grade 8 students
also had a statistically higher average pre-score than Grade 6 (t(38) = 2.59);
p= 0.01). However, there was no statistical dierence between Grades 7 and 8
average pre-scores (t(43) = 1.91; p= 0.06). Grade 6 had a statistically lower
post-score than both Grade 7 (t(55) = 5.72); p< 0.001) and Grade 8
Table 6. Student pre-post results on VELA assessment.
pre Mean (SD) post Mean (SD) tp-value Cohensd
Grade 6 (n= 26) 41.3 (17.6) 53.2 (20.7) 3.55 0.00 0.65
Grade 7 (n= 31) 65.5 (13.8) 80.0 (14.0) 6.13 0.00 1.02
Grade 8 (n= 14) 56.3 (17.4) 68.5 (18.9) 4.27 0.00 0.67
(t(38) = 2.28; p= 0.03). Grade 8 also had a statistically lower average post-
score than Grade 7 (t(43) = 2.22; p= 0.03).
Although all our analyses presented in this paper include all items of the pre-
post assessment, it is worth pointing out that one item on the pre-posttest
(Item#2) was found to be problematic as the language of the question was
ambiguous. This occurred due to a last-minute change to the question which
resulted in the item being used for the VELA intervention without being tested in
classrooms prior to its use. The pre- and post-scores for the item for all three
grades were very low, and the diculty level was 0.35 (the highest diculty
among all the items in the assessment). Item#2 scores normalized as
SD = 18.8 and post-score: M=20.8,SD = 29.1; Grade 7 pre-score: M=44.5,
SD = 30.0 and post-score: M=46.5,SD = 35.2; Grade 8 pre-score: M=34.3,
SD = 28.7 and post-score: M=40.0,SD = 40.0. The item has since undergone
revisions and is currently being eld-tested with various student populations
studying introductory programming in Scratch.
There was no statistically signicant dierence in average pre-score
between females (M= 58.7; SD = 18.3) and males (M= 52.3; SD = 19.5); t
(69) = 1.38; p= 0.17. Though females scored higher than males on average in
the posttest, there was no statistically signicant dierence in the average
post-score either between females (M= 73.5; SD = 20.3) and males (M= 64.0;
SD = 21.0); t(69) = 1.88; p= 0.06. Similarly, average learning gains were not
statistically dierent for females (M= 14.8; SD = 12.3) and males (M= 11.7;
SD = 15.1); t(69) = 0.91; p= 0.37.
Prior math score was a signicant predictor of pre-score; English was not
(Table 7). However, when controlling for pre-score, neither math nor ELA were
signicant predictors of post-score (Table 8). Since the distribution of students
by ethnicity was uneven and sample sizes very small for most groups, we could
not perform any meaningful analyses by ethnicity. Learning gain was not
statistically dierent by or predicted by gender (as mentioned above), grade
(F(2, 68) = 0.20; p= 0.12), or prior math or English score (Table 9)i.e. the
Table 7. Regression to test predictors of pre-score (N= 71; R2 = 0.51).
Coe. Std. Err. t P>t
Prior English Score 2.58 2.79 0.93 0.36
Prior Math Score 9.02 2.49 3.63 0.00
Intercept 20.17 6.49 3.11 0.00
Table 8. Regression to test predictors of post-score (N= 71; R2 = 0.66).
Coe. Std. Err. t P>t
Pre-score 0.57 0.10 5.42 0.00
Prior English Score 4.22 2.34 1.80 0.08
Prior Math Score 3.44 2.33 1.48 0.14
Intercept 13.73 5.02 2.74 0.01
intervention showed no statistically signicant dierence in learning gains in
students by gender or prior academic preparation, and all grades showed
statistically similar learning gains.
We nd this last nding to be very signicant in that it suggests that learning gains
were not linked to specic grades or reported student characteristics. It demonstrates
the promise of the VELA approach and curriculum to reach all students regardless
of prior preparation. Also, prior math scores losing signicance as predictor of
post-score when controlling for pre-score is a departure from the Grover (2014)
nding where prior math was a signicant predictor of the post-score even when
controlling for the pre-score. While one would have expected Grade 8 to perform
better than Grade 7 (who would be expected to outperform Grade 6), our
selections of schools and classrooms (teachers) made our results seem less
surprising. Grades 6 and 8 were low performing schools, and as a result, started
out lower and ended up lower than Grade 7 even though all three grades showed
signicant gains. The pace of the curriculum, as well as lack of strategies and
curricular materials for dierentiation across grades and abilities also meant that
the pre-posttest as well as the curriculum may have been an unduly heavy lift for
younger students and those with poor prior academic preparation in low-
performing schools. This is discussed further in the Survey and Interviews analysis
Scratch projects
Qualitative analyses of 54 open-ended Scratch projects by VELA students and
comparisons with the dataset (of N= 60 projects) from other district class-
rooms revealed that learners in our intervention clearly demonstrated better
facility with the introductory programming concepts in their Scratch projects
their programs were more complex as compared to the comparison sample of
projects collected from non-VELA study students and demonstrated better use
of VELA constructs. Our salient ndings from the comparison of Scratch
projects are as follows.
As seen in Table 10, studentsaverage total scores were signicantly higher
for VELA studentsprojects. Although average scores for general factors,
mechanics of design, and user experiencewere not statistically dierent
across the two groups, VELA studentsprojects scored statistically signicantly
higher on basic coding constructsand advanced constructs. VELA students
Table 9. Regression to test predictors of learning gain (dierence from pre-
score to post-score) (N= 71; R2 = 0.04).
Coe. Std. Err. t P>t
Prior English Score 3.06 2.59 1.18 0.24
Prior Math Score 0.44 2.37 0.19 0.85
Intercept 5.27 5.09 1.04 0.30
average use of Boolean operators in their free-choice projects was also statisti-
cally signicantly higher than that of non-VELA students. This is a signicant
nding, as our work consciously introduced this key concept that is often
given little to no attention in middle school (Grover et al., 2018). Variables,
Repeat-untilloops, and Relational operators, also showed higher use on
average in VELA projects than non-VELA projects, however, the dierence was
not statistically signicant. Another interesting nding was the spread of
projects that used Variables,Repeat-untilloops, Relational operatorsand
Boolean Operators. The number of VELA projects (as a percentage of N=54)
was signicantly higher than the number of non-VELA projects (as
a percentage of N= 60) for the use of each of those foundational constructs
and relevant focal VELA concepts (Table 11). Lastly, grade-wise comparisons
(Table 12) showed that, in general, grade 7 projects signicantly outperformed
the non-VELA projects. Also, grade 6 VELA projects showed signicantly more
use of Basic coding constructsand Boolean Operatorsthan non-VELA grade
6 projects.
VELA studentsresults on this comparative evaluation of nal Scratch projects are
very encouraging, and especially their use of constructs related to our focal con-
cepts. We realize that examining open-ended projects have their pros and cons.
Open-ended projects do not always tell the whole story of student learning as
studentsfree-choice projects may not showcase all that they have learned, or
students may have received help and used code/constructs that they do not
Table 10. Two-sample t-tests of average scores on VELA and the non-VELA comparison nal,
open-ended projects of choice (gray rows indicate statistically signicant dierences).
Mean (SD)
(N= 54)
Non-VELA Mean (SD)
(N= 60) tp-value Cohensd
Total Score 96.0 (53.1) 71.8 (40.0) 2.8 0.01 0.52
General Factors 31.4 (6.3) 29.2 (10.9) 1.2 0.22 0.23
Mechanics of Design 17.5 (14.1) 16.4 (14.2) 0.4 0.67 0.08
User Experience 4.6 (4.8) 3.4 (3.5) 1.5 0.93 0.29
Basic Coding Constructs 31.83 (25.2) 18.17 (15.2) 3.5 0.00 0.67
Advanced Constructs 10.72 (16.9) 4.6 (8) 2.5 0.01 0.47
Variables 1.3 (2.9) 0.9 (1.0) 1.6 0.10 0.31
Boolean Operators 2.8 (4.9) 0.9 (2.5) 2.6 0.01 0.50
Relational Operators 4.3 (4.4) 2.7 (4.4) 1.9 0.06 0.36
Repeat-Until Loops 1.7 () 1.1 (2.1) 1.3 0.21 0.24
Table 11. Percentage of VELA and non-VELA projects that used VELA constructs.
VELA Projects that used
these constructs
Non-VELA Projects that used
these constructs
VELA % (out
of 54)
Non-VELA %
(out of 60)
Variables 40 29 74% 48%
22 14 41% 23%
40 30 74% 50%
20 17 37% 28%
understand well (Kurland & Pea, 1985). Finished projects, as Grover et al. (2017)
explain, also do not show important aspects of learning that can be understood
only from in-process evidence, for example how student debugged or approached
the construction of a computational solution. On the ip side, a free-choice project
showcases the complexity of constructuseandprogrammingstructurethat
astudentchooses to incorporate when no constraints have been imposed. Open-
ended projects are considered important measures of learning and constitute an
authentic form of assessment since children learn deeply when they create pro-
ducts with personal meaning that require understanding and application of knowl-
edge (Barron & Darling-Hammond, 2008). Also, design activity involves stages of
revisions as students dene, create, assess, and redesign their products, and it often
benets from collaboration between students (Grover et al., 2015). Examining these
artifacts thus bears the potential of providing valuable insight into the student
learning experiences in introductory programming that can also point the way
toward areas for improvement in introductory programming curricula (Grover et al.,
2018). Additionally, our evaluation of the projects was very exhaustive. Even though
construct frequency as a measure is not always adequate (Meerbaum-Salant et al.,
2011), the rubric accommodates for duplication of code, and caps frequency count
for most constructs in addition to accounting for several constructs only through
presence or absence (1 or 0).
Lastly, grade-wise comparisons of the projects between the VELA and non-
VELA sample groups provided insight into specic topics that VELA students
learned better. Given our choice of schools and classrooms, grade 7 outperform-
ing the non-VELA group was not entirely unexpected. However, it is signicant
that grade 6 VELA students had signicantly higher scores for basic coding
constructsas well as use of Boolean operatorscompared to the non-VELA
grade 6 group given that grade 6 (and grade 8) VELA classrooms comprised
several struggling learners compared to the average grade 6 or 8 classroom in
the district.
Table 12. Grade-wise comparison of VELA and non-VELA projects.
Grade 6
Grade 7
Grade 8
Grade 6
Grade 7
Grade 8
VELA Sample (N= 54) Non-VELA Sample (N= 60)
Total Score 75.4 (35.0) 137.0*** (54.8) 63.4 (26.6) 70.9 (46.0) 75.9 (36.0) 66.8 (16.6)
General Factors 30.6 (6.8) 34.5* (6.1) 27.9 (3.4) 29.9 (12.8) 28.7 (9.1) 28.0 (3.8)
Mechanics of Design 10.9 (7.8) 28.9** (15.5) 9.5 (4.4) 17.8 (17.6) 15.4 (7.8) 12.2 (8.0)
User Experience 1.8 (2.2) 9.3*** (4.0) 1.4 (1.7) 3.0 (3.5) 4.3 (3.7) 3.3 (3.0)
Basic Coding Constructs 25.3* (15.3) 46.3** (30.9) 19.4 (14.8) 16.1 (16.2) 21.4 (14.9) 19.5 (10.6)
Advanced Constructs 6.8 (9.1) 18.0* (23.2) 5.2 (9.3) 4.0 (7.2) 6.2 (10.3) 3.8 (6.2)
Variables 1.3 (0.8) 1.3 (2.1) 1.1 (1.0) 1.0 (1.1) 0.61 (1.0) 1.0 (1.1)
Boolean Operators 2.6 (4.7) 4.7 (6.0) 0.2 (0.4) .44 (1.7) 2.0 (3.8) 0.1 (0.4)
Relational Operators 3.4 (3.9) 6.5 (4.8) 2.3 (2.8) 2.0 (3.8) 4.4 (5.5) 2.0 (3.7)
Repeat-Until Loops 1.4* (2.1) 1.9 (3.6) 1.7 (2.9) 0.8 (1.8) 1.5 (2.8) 1.0 (1.8)
Note: Asterisks indicate signicant dierence between the score of VELA versus non-VELA group in the same
grade (* p < .05; ** p < .01; *** p < .001)
Surveys and interviews
These aforementioned ndings notwithstanding, the range of studentsreadi-
ness to engage and learn, their instructional needs, and individual student
reactions, varied considerably both within and across the three classrooms.
The 6th-grade students had the most challenges, whereas the high-achieving
7th-grade class demonstrated most engagement with VELA ideas and
This was evidenced in studentspost-survey feedback on the digital activ-
ities (Table 13 and Figure 10). Overall, students found Cats and Ladders to be
most engaging and fun. Graphical Looping was found to be too simple and
Table 13. Teacher and student reactions to some of the VELA digital activities.
Reactions to Cats and
(Teacher): maybe it is having that tool that reaches the student on so many levels
being that it is animated, there is sort of an emotional interest, all of that, so that
story sticks out better. And so it may not really have huge payoin that it shows
you how to build an expression, but as the teacher emphasizes over and over
again now that expressions are made from variables and operators and how that
comes up later, it refers back to cats and ladders it is like oh yeah, cats and
ladders, the two ladders, the two ladders. So the power becomes more on how
the teacher refers back to the tool in later discussion in referencing the story
(7th grade student): I thought it was interesting because this happens in real life,
and I like cats, and I like the idea of saving them..I thought it was more
challenging and interesting because you would have to not only type in the blue
ladder height and orange ladder height, but also which building it was in. It
made it more connected to realize because in the city that I live in, theres a lot
of buildings.
(8th grade student):Its. . . to show us about height. And like the variables change.
Because like you could change the numbers. . .It shows us variables. Like how
high the ladder could go, like how we could change it.
(6th grade student): Cats & Ladders was fun and cool(a few other students had
similar reactions)
(7th grade student): I liked cats and ladders because it was like a real game.
Reactions to Graphical
(8th grade student): Its sort of like a puzzle. . .It teaches you the purpose of what
loops do. . .It reminded like if we go to a grocery store and we could come back
and get more food from there and then come back. ..Like each day like you
like. . . it can be from like school, you go back home. And from the home you go
back to school
(7th grade): Graphical Looping was too easy and boring(a few other students
had similar reactions)
Reactions to Alarm
(6th grade student): I did not really understand it(a few students had similar
(8th grade student): I liked alarm clock the most because it was most challenging
(8th grade student):It was most interesting
(6th grade student): It took too long
(Teacher): And then the Boolean switch and the alarm clock one, I think are too
big for the kids, as an activity to handle on their own. And so they become very
teacher directed, and along that way you lose a lot of kids.
(Teacher): The interface is confusing, in particular with regard to drawing attention
to which features are more important than others. . . Some kids might get it and
understand what to ignore and what to work with, but its not clear at rst. And
so even some of the naming schemes for what the boxes are, what the tables
are, this table is how youre going to use this, and this table is how youre going
to use that.
(Teacher): Imean I think its a really cool project idea, but I think maybe just
changing up the way kids enter in variables and expressions and stu.
lacking in challenge. Some students found a few aspects of Alarm Clock to be
confusing, although others indicated they learned from it.
Additionally, while students engaged well with variables, arithmetic expres-
sions, and loops, there was relatively a greater struggle with abstraction and
with Boolean variables and expressions (Figure 10). Some of these struggles
appear to be related to experiences with the activities. Teachers noted in
interviews, for example, that the expression-authoring features in the Alarm
Clock and Three Switches microworld was complex and confusing for some
students. However, one teacher also commented that although she found
good bridging in Scratch lessons to VELA activities on variables, looping and
conditionals, the connections to the expression abstraction work in Alarm
Clock were not suciently revisited or reinforced in the subsequent Scratch
programming activities. That said, one teacher commented on the deep value
she saw in the VELA curriculum
Well for me, I felt like it really educated me quite a bit, deeply and powerfully. And so it was
classroom, but the content itself. So it developed my content knowledge really well. And
Imexcitedtonowkindofapplyitintheclassroom, and try to marry our current curriculum
to some of these ideas and esh it out more. . .. I feel that this group learned programming
deeper, by about 50% or more, if I could put a value on it, than previous groups Ive taught,
and so that their understanding of programming environments and Scratch is more robust
than previous groups. And itsjustanintuitionofseeingit.(VELATeacher)
Figure 10. Survey feedback from students on their understanding of VELA concepts.
Conclusion and next steps
Our current work in the context of middle school introductory programming draws
on an approach in mathematics education that has succeeded in achieving better
conceptual learning in middle school for diverse students. We created and exam-
ined dynamic digital interactives and microworlds that engage students with hard-
to-learn programming concepts, nding particularly pedagogic value of dynamic
representations to variability and abstraction in computing. We also drew from
learning sciences research to scaold novice learnersguided inquiry and explora-
tion of programming concepts in a context separate from programming before
they employ them in programs. Our empirical research suggests that these theories
pertaining to conceptual learning (drawn from research outside of CS) can be put to
work with success to aid the novice programmersengagement with and learning
of hard-to-learn concepts foundational to programming.
Our activity designs reect several themes. For example, the importance of
naming of choosing names that are both meaningful to the problem context in
which they are relevant, and productive inwhateverlogicalorsymbolicorverbal
grammar will be used to reference and manipulate them is one such essential
theme, and recurs in our activities on variables and expressions (where names stand
for changing quantities and for relationships between changing quantities), as well
as on abstraction (where names stand as summaries for exhaustive denitions and
detail). Another is that story is a productive and universally accessible metaphor and
proxy for program code in a context in which learners are not yet able to grapple with
code explicitly. Stories conveyed orally, through comic strips, or in writing mirror
typical code in having well-dened execution ows which are primarily sequential
but occasionally undertake repetitive loops, and other control ows. Finally,
dynamic representation design principles oer a ladder to abstract ideas and
concepts emerging from concrete and manipulable representations that students
nd compelling.
Our results demonstrate the promise of introductory experiences with
dynamic representations and relevant storyexamples and contexts for con-
ceptual exploration. Overall, our students demonstrated learning gains in
understanding of the VELA concepts as evidenced by pre-post assessment
data analyses and an examination of nal free-choice Scratch projects. More
importantly, these learning gains were not signicantly dierent by prior math
preparation or gender or ethnicity. Our studentsopen-ended, free-choice
projects scored signicantly better on key dimensions (those more closely
related to our focal foundational programming concepts) than projects from
a comparison group.
This empirical research demonstratesthepromiseofournovelapproachand
curriculum for providing conceptual supports to novice programmers. Student and
teacher feedback on the VELA activities as well as deeper item analyses of the
assessment suggest areas for improvement in the next iteration of this DBR. Our
next steps also involve improving the activity designs and building more robust
curricular bridges between VELA and Scratch activities. Additionally, the role of the
classroom teachers in the enactment of this curriculum (based on their own
expertise and experience in teaching as well as conceptual understanding of CS)
also plays an important role in the impact of a curriculum on student learning.
Related ongoing research is examining individual student and teacher cases to
better understand the learning process andtheimpactofdiversebackgroundsand
classroom environments. Future research will examine how specicelementsof
VELA activities contribute to student understandings for a more complete picture of
the mechanics of learning in our novel approach to teaching introductory program-
ming. We are also developing teacher professional development modules around
the VELA curriculum to support deeper conceptual understanding of introductory
programming concepts for teachers.
We thank the National Science Foundation for grant funding support (NSF #1543062). We
are grateful to Jeremy Roschelle, Satabdi Basu, Matthias Hauswirth, and Bryan Twarek for
their assistance and intellectual contributions, and to research staat SRI International as
well as teachers and students for their participation in this research.
Disclosure statement
No potential conict of interest was reported by the authors.
This work was supported by the National Science Foundation, USA, under Grant 1543062.
Notes on contributors
Shuchi Grover, Ph.D., a computer scientist and learning sciences researcher by training, is
Chief Learning Scientist at Looking Glass Ventures in Palo Alto, California. Formerly senior
research scientist at SRI International, Dr. Grovers current research focuses on computer
science education especially introductory programming and the development of computa-
tional thinking skills in preK-12 learners.
Nicholas Jackiw is a senior research scientist at the Center for Education Research and
Innovation, SRI International working in mathematics and computer science education. He
also designs and develops software for learners of mathematics, where his work includes
The Geometers Sketchpad and TouchCounts.
Patrik Lundh, Ph.D., is a senior education researcher at the Center for Education Research
and Innovation, SRI International. Dr. Lundh conducts research in K-12 science, mathematics,
and computer science education in school and afterschool settings. He has extensive
experience in qualitative research methods, design research, curriculum implementation
and evaluation research.
Shuchi Grover
Ackermann, E. (2001). Piagets constructivism, paperts constructionism: Whats the dier-
ence. Future of Learning Group Publication,5(3), 438.
Barron, B., & Daring-Hammond, L. (2008). How can we teach for meaningful learning? In
L. Daring-Hammond, B. Barron, P. D. Pearson, A. H. Schoenfeld, E. K. Stage,
T. D. Zimmerman, . . . J. L. Tilson (Eds.), Powerful learning: What we know about teaching
for understanding (pp. 1-10). San Francisco: Jossey-Bass.
Bransford, J. D., Brown, A., & Cocking, R. (2000). How people learn: Mind, brain, experience,
and school. Washington, DC: National Research Council.
Bau, D., Gray, J., Kelleher, C., Sheldon, J., & Turbak, F. (2017). Learnable programming: blocks
and beyond. Communications of the ACM,60(6), 7280.
Du Boulay, B. (1986). Some diculties of learning to program. Journal of Educational
Computing Research,2(1), 5773.
Ebrahimi, A. (1994). Novice programmer errors: Language constructs and plan composition.
International Journal of Human-Computer Studies,41(4), 457480.
Franklin, D., Skifstad, G., Rolock, R., Mehrotra, I., Ding, V., Hansen, A., . . . Harlow, D. (2017,
March). Using upper-elementary student performance to understand conceptual sequen-
cing in a blocks-based curriculum. In Proceedings of the 2017 ACM SIGCSE Technical
Symposium on Computer Science Education (pp. 231236), Seattle, WA, USA. ACM.
Gobil, A. R. M., Shukor, Z., & Mohtar, I. A. (2009). Novice diculties in selection structure. In
International Conference on Electrical Engineering and Informatics, 2, 351356. New
Jersey, USA: IEEE.
Gray, D., Kelleher, J., Sheldon, C., & Turbak, F. (2017). Learnable programming: Blocks and
beyond. Communications of the Acm,60(6), 7280.
Grover, S. (2014). Foundations for advancing computational thinking: balanced designs for
deeper learning in an online computer science course for middle school students. Stanford:
Stanford University
Grover, S. (2019). An assessment for introductory programming concepts in middle school
computer science. In Proceedings of the 2019 Annual Meeting of the National Council on
Measurement in Education (NCME), Toronto, CA.
Grover, S., & Basu, S. (2017). Measuring student learning in introductory block-based
programming: examining misconceptions of loops, variables, and boolean logic. In
Proceedings of the 48th ACM Technical Symposium on Computer Science Education
(SIGCSE 17). Seattle, WA: ACM.
Grover, S., Basu, S., Bienkowski, M., Eagle, M., Diana, N., & Stamper, J. (2017). A framework for
using hypothesis-driven approaches to support data-driven learning analytics in measur-
ing computational thinking in block-based programming environments. ACM
Transactions on Computing Education (TOCE),17(3), 14.
Grover, S., Basu, S., & Schank, P. (2018). What we can learn about student learning from
open-ended programming projects in middle school computer science. In Proceedings of
the 49th ACM Technical Symposium on Computer Science Education (pp. 9991004),
Baltimore, MD, USA. ACM.
Grover, S., Jackiw, N., & Lundh, P. (n.d.). Non-programming digital interactives to advance
learning of introductory computing concepts for diverse student populations. Interactive
Learning Environments.
Grover, S., Lundh, P., & Jackiw, N. (2019). Non-programming activities for engagement with
foundational concepts in introductory programming. In Proceedings of the 50th ACM
Technical Symposium on Computer Science Education, Minneapolis, MN, USA. ACM.
Grover, S., & Pea, R. (2014). Expansive framing and preparation for future learning in
middle-school computer science. In International Conference of the Learning Sciences
Conference, Boulder, CO, USA. ISLS.
Grover, S., Pea, R., & Cooper, S. (2015). Designing for deeper learning in a blended computer
science course for middle school students. Computer Science Education,25(2), 199237.
Grover, S., Pea, R., & Cooper, S. (2016). Factors inuencing computer science learning in
middle school. In Proceedings of the 47th ACM Technical Symposium on Computer Science
Education. Memphis, TN: ACM.
Jackiw, N. (19912009). The geometers sketchpad (computer software V1, V5). Emeryville, CA:
Key Curriculum Press.
Kurland, D., & Pea, R. (1985). Childrens mental models of recursive LOGO programs. Journal
of Educational Computing Research,1(2), 235243.
Laborde, J. M., & Strässer, R. (1990). Cabri-géomètre: A microworld of geometry for guided
discovery learning. Zentralblatt für didaktik der athematic,90(5), 171177.
Lahtinen, E., Ala-Mutka, K., & Järvinen, H. M. (2005). A study of the diculties of novice
programmers. ACM SIGCSE Bulletin,37(3), 1418.
Lewis, C. M., & Shah, N. (2012). Building upon and enriching grade four mathematics
standards with programming curriculum. In Proceedings of the 43rd Technical
Symposium on CS Education. Raleigh, NC: ACM.
Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning?
American Psychologist,59(1), 1419.
Meerbaum-Salant, O., Armoni, M., & Ben-Ari, M. (2011, June). Habits of programming in
scratch. In Proceedings of the 16th annual joint conference on Innovation and technology in
computer science education (pp. 168172), Darmstadt, Germany. ACM.
Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered
design. ETS Research Report Series,2003(1), i-29.
Papert, S. (1991). Situating constructionism. In I. Harel & S. Papert (Eds.), Constructionism (pp.
1-11). Ablex Publishing.
Pea, R., & Kurland, D. (1984). On the cognitive eects of learning computer programming.
New Ideas in Psychology,2, 137168.
Pears, A., Seidman, S., Malmi, L., Mannila, L., Adams, E., Bennedsen, J., ., . .. Paterson, J. (2007).
A survey of literature on the teaching of introductory programming. In Proceedings of the
38th Technical Symposium on CS Education, Covington, KY, USA. ACM.
Robins, A., Rountree, J., & Rountree, N. (2003). Learning and teaching programming:
A review and discussion. Computer Science Education,13(2), 137172.
Roschelle, J., Shechtman, N., Tatar, D., Hegedus, S., Hopkins, B., Empson, S., . .. Gallagher, L. P.
(2010). Integration of technology, curriculum, and professional development for advan-
cing middle school mathematics: Three large-scale studies. American Educational Research
Journal,47(4), 833878.
Samurcay, R. (1989). The concept of variable in programming: Its meaning and use in
problem-solving by novice programmers. In E. Soloway & J. C. Spohrer (Eds.), Studying
the novice programmer (pp. 161178). Psychology Press.
Schattschneider, D., & King, J. (1997). Geometry turned on: Dynamic software in learning,
teaching, and research. Washington: MAA.
Schoeld, E., Erlinger, M., & Dodds, Z. (2014). MyCS: CS for middle-years students and their
teachers. In Proceedings of the 45th ACM technical symposium on Computer science
education, Atlanta, GA, USA (pp. 337342). ACM.
Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction,16(4),
Shute, V. J. (1991). Who is likely to acquire programming skills? Journal of Educational
Computing Research,7(1), 124.
The White House. (2016). Computer science for all. Retrieved from
Wang, F., & Hannan, M. J. (2005). Design-based research and technology-enhanced learn-
ing environments. Educational Technology Research and Development,53(4), 523.
Wiggins, G. P., & McTighe, J. (2005). Understanding by design. Association for Supervision &
Curriculum Development.
Sample item from design VELA pre-post assessment
... Algorithm concepts (Table 1) include branching (i.e., if/then/else conditional statements), iteration (i.e., loops and repeating steps), methods (i.e., encapsulated sequence of steps), and variables (i.e., values that can change). These concepts are commonly referenced in CT literature (Peel et al., 2015;Peel & Friedrichsen, 2018;Brennan & Resnick, 2012;Grover et al., 2019;Lye & Koh, 2014). CT practices are actions, or things students do during algorithm creation and other computational activities, whereas concepts are represented as written aspects of an algorithm. ...
... The CT-AE instructional approach aligns with other unplugged literature showing that unplugged CT lessons can support programming skills and learning (Grover et al., 2019;Hermans & Aivaloglou, 2017). While these articles used unplugged lessons that supported programming, they were not situated in science contexts. ...
Full-text available
Computing has become essential in modern-day problem-solving, making computational literacy necessary for practicing scientists and engineers. However, K–12 science education has not reflected this computational shift. Integrating computational thinking (CT) into core science courses is an avenue that can build computational literacies in all students. Integrating CT and science involves using computational tools and methods (including programming) to understand scientific phenomena and solve science-based problems. Integrating CT and science is gaining traction, but widespread implementation is still quite limited. Several barriers have limited the integration and implementation of CT in K–12 science education. Most teachers lack experience with computer science, computing, programming, and CT and therefore are ill-prepared to integrate CT into science courses, leading to low self-efficacy and low confidence in integrating CT. This theoretical paper introduces a novel instructional approach for integrating disciplinary science education with CT using unplugged (computer-free) activities. We have grounded our approach in common computational thinking in STEM frameworks but translate this work into an accessible pedagogical strategy. We begin with an overview and critique of current approaches that integrate CT and science. Next, we introduce the Computational Thinking through Algorithmic Explanations (CT-AE) instructional approach. We then explain how CT-AE is informed by constructionist writing-to-learn science theory. Based on a pilot implementation with student learning outcomes, we discuss connections to existing literature and future directions.
... Analysis of the interaction effects of approach to learning programming and students' characteristics showed that prior programming experience and gender had neither significant interaction nor main effect on CT and programming knowledge. These findings are consistent with prior studies that found no effect of gender on programming (Grover et al., 2019;Lau & Yuen, 2009Sullivan & Bers, 2019). ...
Full-text available
Most studies suggest that students develop computational thinking (CT) through learning programming. However, when the target of CT is decoupled from programming, emerging evidence challenges the assertion of CT transferability from programming. In this study, CT was operationalized in everyday problem-solving contexts in a learning experiment ( n = 59) that investigated whether learning programming enhances students’ CT skills. Specifically, this study examined the influence of a novel, systematic and micro instructional strategy that is grounded in abstraction and comprised of four independent but related processes – discover, extract, create, and assemble (DECA) towards simplification of problem-solving. Subsidiary questions explored the effects of students’ age, gender, computer proficiency, and prior programming experience on the development of CT. No significant difference was found between the CT skill and programming knowledge of the groups at the posttest. However, within-group paired t-tests showed that the experimental group that integrated DECA had significant improvement in CT but not in the control group across the pretest-posttest axis. Implications of the inconclusive finding about the transfer of programming skills to CT are emphasized and the arguments for disentangling CT from programming are highlighted.
... This disarray could be problematic because a systematic learning approach is crucial when studying computer programming. Specifically, novice programmers must understand the underlying concepts first (e.g., basic syntax, data types, variables, operators, expressions) before coding complex algorithms (Grover et al., 2019). The inclination of TikTokers to produce videos beneficial to one's programming career likewise implies that the content is more helpful to industry practitioners than students. ...
Conference Paper
Full-text available
Recent studies have acknowledged social media as a valuable pedagogical tool for connecting the formal and informal learning gap. However, as a new platform, the literature is sparse on the potential of TikTok as a knowledge source. In this study, we explored programming TikTok videos in the #LearnProgramming webpage in terms of content (programming languages and topics) and characteristics (video styles and types). Although TikTok is principally an entertainment destination, our results show that the platform likewise has informative videos. The 349 videos that we examined received a total of 10,046,000 views, 10,523 comments, 932,871 likes, and 35,095 shares, implying extremely high levels of user engagement. Tiktokers showing tips and tricks are the most recurring content type. From a macro perspective, we noticed that TikTokers do not follow the ethos of the platform (e.g., dancing) when producing educational content. This deviation demonstrates the intent of TikTokers to educate than to entertain. Although it is too early to conclude that TikTok can operate as a nanolearning platform, we discovered a substantial amount of content for and engagement from programming learners. Our results lay a potent foundation for devising actionable scholastic implications, policies, and recommendations concerning TikTok consumption. Future works and research prospects were also discussed to propel the social media and nanolearning literature forward.
... Looping is a method of repeating a procedure as many times as predetermined stopping conditions allow (Shouthiri & Thushika, 2018). Since many software applications repeat their work until the desired condition is reached, looping is an essential part of programming (Grover, Jackiw, & Lundh, 2019). With iterations, programmers are not required to write as much program code as they would like (Brown & Wilson, 2018). ...
Full-text available
This study aims to evaluate the effect of the use and development of interactive and visual learning media in learning looping by utilizing a web-based digital platform called Loopers. The research design used a qualitative descriptive with case study approach in-depth interview technique. The subjects of this research are five students of the State Vocational School Pekerjaan Umum, Bandung, Indonesia. We use three phases in conducting research through 1) problem and needs analysis, 2) web design, and 3) implementation and testing. Loopers was developed using the waterfall method. Loopers can demonstrate the implementation of a productive learning process and measure the effectiveness of interactive and visualized learning media. The findings show that Loopers can effectively facilitate experimental demonstrations and written modules. Respondents think this learning media is effective and can increase their interest, motivation, and interest in learning looping material. The next research is to enrich the Loopers feature and conduct trials with larger respondents.
... It is important to note, however, that 'computational thinking' does not necessarily have to take the form of computer coding. Unplugged activities, such as using dice to teach conditionals or short stories to teach variables, have the potential to provide students with deeper understanding of concepts prior to engaging with more complex code, while still providing the benefits of the computational process (Grover et al., 2019). Moreover, computational processes are suited to benefit students' spatial reasoning, as these programs often involve spatial transformations, as well as identifying relationships and patterns (Gouws et al., 2013;Halberda et al., 2008). ...
Full-text available
In this research, we explore the potential for computational thinking (CT) to benefit the spatial reasoning abilities of thirty-three middle-school students (aged 11–13) in grades 6 and 7. There is an increasing focus on the explicit development of spatial reasoning throughout the mathematics curriculum, such as geometry, as students’ success in mathematics is largely tied to their spatial reasoning abilities. CT provides a method through which numerous mathematics concepts can be taught or explored, but also offers an avenue through which spatial reasoning skill can be developed. We used the block programming website Scratch and programmable robotics in weekly ‘coding clubs’ run in two different elementary school classrooms for 10 weeks each. Lessons were structured to develop students’ CT abilities and mathematical reasoning, as well as being tied to their mathematics units such as geometry and number sense. Pre- and post-test data measured student spatial reasoning. Qualitative data in the form of student journal responses and teacher and researcher observations further illustrate incidents of learning which benefitted students’ spatial reasoning abilities. Our results found improvement over time. Implications for classroom instruction and further research will be discussed.
... Abstraction provides the last letter for the acronym VELA, the label of a project aimed at enhancing the understanding of programming concepts in K-12 (Grover et al., 2019) -the other letters standing for Variables, Expressions, and Looping. In the context of this project, abstraction is defined as "the process of giving a name to a specific collection of details as a way of referencing its purpose without quoting or enumerating its detail." ...
When we “think like a computer scientist,” we are able to systematically solve problems in different fields, create software applications that support various needs, and design artefacts that model complex systems. Abstraction is a soft skill embedded in all those endeavours, being a main cornerstone of computational thinking. Our overview of abstraction is intended to be not so much systematic as thought provoking, inviting the reader to (re)think abstraction from different – and perhaps unusual – perspectives. After presenting a range of its characterisations, we will explore abstraction from a cognitive point of view. Then we will discuss the role of abstraction in a range of computer science areas, including whether and how abstraction is taught. Although it is impossible to capture the essence of abstraction in one sentence, one section or a single paper, we hope our insights into abstraction may help computer science educators to better understand, model and even dare to teach abstraction skills.
As we begin to identify essential skills for the future, there has been an increased focus on how to define and implement computational thinking within education. However, the question of how to engage children with coding activities within the context and limitations of the curriculum remains relatively explored. In our study, a human-centered design approach was followed to guide a small group of children through the co-design, implementation and evaluation of a board game. The board game had two versions: an offline version involving a robotic agent and an online version with its digital equivalent. A mixed-method approach was used to understand and compare levels of engagement, usability, and collaboration in both versions of the game. The tangible aspect of working with a robotic agent in the offline version led to greater levels of motivation and involvement with the game mechanics. Tangible robotics games tend to be more approachable for children. Our study shows that using such games by connecting the mathematics curriculum with coding problems can successfully engage participants in collaborative computational thinking activities. We contribute to the field by providing human-centered guidance for the creation of educational tools to develop computational thinking. Such guidance is not only applicable in technologically rich environments; it can also be applied in educational communities with limited access to technology.
The importance of computational thinking (CT) as a 21st-century skill for future generations has been a key consideration in the reforms of many national and regional educational systems. Much attention has been paid to integrating CT into the traditional subject classrooms. This paper describes a scoping review of learning tools for integrating CT and mathematics in current empirical studies published from 2015 to 2021. The review showed that most of the studies implemented CT-intensive Math-connected integration. Five major types of CT tools had been identified, i.e., digital tangibles, apps and games, programming languages, formative or summative assessments, and other technological tools. In many instances, the tools also provide functions of assessment of CT skills. The most assessed CT competencies were including algorithms and algorithmic thinking, abstraction, testing and debugging, loops, and sequences. Geometry and Measurement was the most assessed mathematics topic. Our scoping review is beneficial in the investigation of the literature on CT and mathematics education, as well as guides those who are interested in developing curriculum, programs, or assessments that involve the integration of CT and mathematics.
Full-text available
The aim of this study is to compare the effects of unplugged and plugged-in activities on academic achievement and computational thinking (CT) skills of sixth-grade students. Mixed-method research was carried out to explore whether there were differences between the groups, and to learn the students' opinions and experiences regarding the practices. For the quantitative phase, a quasi-experimental design was used with two groups. For qualitative phase, 12 students were interviewed. The participants were 84 sixth-grade students (between the ages of 10 and 11). The intervention was designed on a selection/construction of activities from seven different basic programming web platforms for the plugged-in group and the proposed national curriculum unplugged activities for the unplugged group. The results showed that significant differences between groups in academic achievement favoring the unplugged activities, but not in CT skills. Development in CT skills contributed to the unplugged group's academic achievement. In addition, qualitative results showed that the plugged-in group perceived their activities as fun and entertaining, but not exactly like a lesson; in contrast, the unplugged group did not experience anxiety or boredom since they perceived the activities as educational. CT explained 27 percent of the variance in academic achievement, suggesting that this skill is important for academic achievement in basic programming. These results suggest that students can improve their academic achievement and maintain the level of CT acquisition across unplugged and plugged-in activities. This article contributes to the body of knowledge about the positive impact of unplugged activities on teaching CT and programming fundamentals.
Existing computational thinking (CT) research focuses on programming in K-12 education; however, there are challenges in introducing it into the formal disciplines. Therefore, we propose the introduction of non-programming plugged learning in mathematics to develop students’ CT. The research and teaching teams collaborated to develop an instructional design for primary school students. The participants were 112 third- and fourth-grade students (aged 9–10) who took part in three rounds of experiments. In this paper, we present an iterative problem-solving process in design-based implementation research, focusing on the implementation issues that lead to the design principles in the mathematics classroom. The computational tasks, environment, tools, and practices were iteratively improved over three rounds to incorporate CT effectively into mathematics. Results from the CT questionnaire demonstrated that the new program could significantly improve students’ CT abilities and compound thinking. The results of the post-test revealed that CT, including the sub-dimensions of decomposition, algorithmic thinking, and problem-solving improved threefold compared to the pre-test between the three rounds, indicating that strengthened CT design enhanced CT perceptions. Similarly, the students’ and teacher’ interviews confirmed their positive experiences with CT. Based on empirical research, we summarize design characteristics from computational tasks, computational environment and tools, and computational practices and propose design principles. We demonstrate the potential of non-programming plugged learning for developing primary school students’ CT in mathematics.
Conference Paper
Full-text available
Teaching of computer science (CS) and programming is rapidly expanding in formal school education. Learning to program is a key ingredient of school CS curricula, and consequently quality measurements of student learning of foundational programming concepts are needed by teachers and researchers. However, high- quality tools for measuring student learning in introductory CS have been under-developed and under-researched. This experience report shares the process of design and refinement of a summative paper- based assessment (that could also be administered online) for introductory programming in middle grades (6-8). We share our experiences with the use of assessment as a pre-post measure in a middle school introductory programming course in diverse, urban school classrooms in the US and use that data to conduct validity, reliability and item discrimination analyses.
Conference Paper
Full-text available
In this paper, we present research on how important features in a curricular activity system can support middle school students’ engagement with and learning of key computer science (CS) concepts. We developed a curriculum with a sequence of non- programming activities intended to support student learning of CS concepts before they start learning programming using block-based approaches.
Conference Paper
Full-text available
Block-based programming environments such as Scratch, App Inventor, and Alice are a key part of introductory K-12 computer science (CS) experiences. Free-choice, open-ended projects are encouraged to promote learner agency and leverage the affordances of these novice-programming environments that also support creative engagement in CS. This mixed methods research examines what we can learn about student learning from such programming artifacts. Using an extensive rubric created to evaluate these projects along several dimensions, we coded a sample of ~80 Scratch and App Inventor projects randomly selected from 20 middle school classrooms in a diverse urban school district in the US. We present key elements of our rubric, and report on noteworthy trends including the types of artifacts created and which key programming constructs are or are not commonly used. We also report on how factors such as students' gender, grade, and teachers' teaching experience influenced students' projects. We discuss differences between programming environments in terms of artifacts created, use of computing constructs, complexity of projects, and use of features of the environment for creativity, interactivity, and engagement. Our findings will help educators of introductory computing be more cognizant of how best to leverage the programming environments they are using, and what aspects they need to focus on as they attempt to address the learning needs of all in "CS For All."
Full-text available
Systematic endeavors to take computer science (CS) and computational thinking (CT) to scale in middle and high school classrooms are underway with curricula that emphasize the enactment of authentic CT skills, especially in the context of programming in block-based programming environments. There is, therefore, a growing need to measure students’ learning of CT in the context of programming and also support all learners through this process of learning computational problem solving. The goal of this research is to explore hypothesis-driven approaches that can be combined with data-driven ones to better interpret student actions and processes in log data captured from block-based programming environments with the goal of measuring and assessing students’ CT skills. Informed by past literature and based on our empirical work examining a dataset from the use of the Fairy Assessment in the Alice programming environment in middle schools, we present a framework that formalizes a process where a hypothesis-driven approach informed by Evidence-Centered Design effectively complements data-driven learning analytics in interpreting students’ programming process and assessing CT in block-based programming environments. We apply the framework to the design of Alice tasks for high school CS to be used for measuring CT during programming.
Conference Paper
Full-text available
Programming in block-based environments is a key element of introductory computer science (CS) curricula in K-12 settings. Past research conducted in the context of text-based programming points to several challenges related to novice learners' understanding of foundational programming constructs such as variables, loops, and expressions. This research aims to develop assessment items for measuring student understanding in introductory CS classrooms in middle school using a principled approach for assessment design. This paper describes the design of assessments items that were piloted with 100 6th, 7th, 8th graders who had completed an introductory programming course using Scratch. The results and follow-up cognitive thinkalouds indicate that students are generally unfamiliar with the use of variables, and harbor misconceptions about them. They also have trouble with other aspects of introductory programming such as how loops work, and how the Boolean operators work. These findings point to the need for pedagogy that combines popular constructionist activities with those that target conceptual learning, along with better professional development to support teachers' conceptual learning of these foundational constructs.
Conference Paper
Full-text available
As more elementary schools commit to integrating computer science instruction into their curricula, they seek guidance on what concepts are appropriate for students at different grade levels. Currently, little is known about how best to sequence computer science learning across elementary grades. In this paper, we present an analysis of 123 students' (age 9-12, grades 4-6) activities in a curriculum implemented in a visual block-based programming language. The goal of this work is to better understand the developmental appropriateness of foundational computer science ideas. All 4th, 5th, and 6th grade students in a single school completed the first module of a curriculum during the same school year with the same instructor. We analyzed each task students attempted and found that for simple concepts, there was little difference in performance between grade levels. However, differences were found for more complex topics, such as whether they completed initialization tasks and the way in which they solved 2-d navigation tasks. A closer look revealed that students understood the basic concepts, but were challenged by deeper applications of the basic concepts and influenced by non-computer science skills. This work serves as an empirically grounded investigation of elementary computer science learning and contributes to our understanding of computer science learning trajectories and concept sequencing in the late elementary grades.
Conference Paper
Full-text available
In this paper, we describe research conducted around a 7-week curriculum designed to introduce middle school students to computer science with a focus on algorithmic thinking and programming. The pedagogical ideas employed in this curriculum were drawn from past research. Empirical investigations over two studies in a public middle school in the US examined changes in students' understanding of algorithmic constructs and the factors affecting that learning. Multi-level analyses revealed that students in both studies (1) achieved substantial learning gains in algorithmic thinking skills and significant growth towards a more mature understanding of computing as a discipline, and (2) found certain CT ideas and constructs more difficult than others. Prior computing experiences and math and English ability were found to be predictors of learning outcomes. Extracurricular experiences with technology also appeared to impact outcomes.
Conference Paper
Programming-in a text-based or visual language-is a complex activity that novices find difficult to learn. In this paper, we present an experience report on how novel activities designed for early conceptual exploration can support middle school students' engagement with and learning of foundational programming concepts beginning programmers often struggle with, specifically, variables, expressions (Boolean, arithmetic, relational), loops, and abstraction. We drew on mathematics education research on dynamic representations to design and develop a set of four non-programming, (open access) web-based, interactive activities and micro-worlds and two unplugged activities as part of a suite of activities that can be embedded in a typical introductory programming curriculum to introduce learners to the target concepts before they encounter them in programming. This report describes the activities as well as our experience with their use as part of an introductory computer science course in three diverse, urban, middle school classrooms in the US
Blocks-based programming has become the lingua franca for introductory coding. Studies have found that experience with blocks-based programming can help beginners learn more traditional text-based languages. We explore how blocks environments improve learnability for novices by 1) favoring recognition over recall, 2) reducing cognitive load, and 3) preventing errors. Increased usability of blocks programming has led to widespread adoption within introductory programming contexts across a range of ages. Ongoing work explores further reducing barriers to programming, supporting novice programmers in expanding their programming skills, and transitioning to textual programming. New blocks frameworks are making it easier to access a variety of APIs through blocks environments, opening the doors to a greater diversity of programming domains and supporting greater experimentation for novices and professionals alike.
Scratch is a visual programming environment that is widely used by young people. We investigated if Scratch can be used to teach concepts of computer science (CS). We developed learning materials for middle-school students that were designed according to the constructionist philosophy of Scratch and evaluated them in a few schools during two years. Tests were constructed based upon a novel combination of the revised Bloom taxonomy and the Structure of the Observed Learning Scratch is a visual programming environment that is widely used by young people. We investigated if Scratch can be used to teach concepts of computer science (CS). We developed learning materials for middle-school students that were designed according to the constructionist philosophy of Scratch and evaluated them in a few schools during two years. Tests were constructed based upon a novel combination of the revised Bloom taxonomy and the Structure of the Observed Learning Outcome taxonomy. These instruments were augmented with qualitative tools, such as observations and interviews. The results showed that students could successfully learn important concepts of CS, although there were problems with some concepts such as repeated execution, variables, and concurrency. We believe that these problems can be overcome by modifications to the teaching process that we suggest. Outcome taxonomy. These instruments were augmented with qualitative tools, such as observations and interviews. The results showed that students could successfully learn important concepts of CS, although there were problems with some concepts such as repeated execution, variables, and concurrency. We believe that these problems can be overcome by modifications to the teaching process that we suggest.