Conference PaperPDF Available

A Dataset of Scratch Programs: Scraped, Shaped and Scored

Authors:

Abstract and Figures

Scratch is increasingly popular, both as an introductory programming language and as a research target in the computing education research field. In this paper, we present a dataset of 250K recent Scratch projects from 100K different authors scraped from the Scratch project repository. We processed the projects' source code and metadata to encode them into a database that facilitates querying and further analysis. We further evaluated the projects in terms of programming skills and mastery, and included the project scoring results. The dataset enables the analysis of the source code of Scratch projects, of their quality characteristics, and of the programming skills that their authors exhibit. The dataset can be used for empirical research in software engineering and computing education.
Content may be subject to copyright.
Delft University of Technology
Software Engineering Research Group
Technical Report Series
A Dataset of Scratch Programs: Scraped,
Shaped and Scored
Efthimia Aivaloglou, Felienne Hermans, Jesus Moreno-Leon,
and Gregorio Robles
Report TUD-SERG-2017-007
TUD-SERG-2017-007
Published, produced and distributed by:
Software Engineering Research Group
Department of Software Technology
Faculty of Electrical Engineering, Mathematics and Computer Science
Delft University of Technology
Mekelweg 4
2628 CD Delft
The Netherlands
ISSN 1872-5392
Software Engineering Research Group Technical Reports:
http://www.se.ewi.tudelft.nl/techreports/
For more information about the Software Engineering Research Group:
http://www.se.ewi.tudelft.nl/
Note: Accepted for publication in the Proceedings of The 14th International Conference on Mining Soft-
ware Repositories, May 20-21, 2017. Buenos Aires, Argentina. IEEE, ACM.
c
2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all
other uses, in any current or future media, including reprinting/ republishing this material for advertising or
promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse
of any copyrighted component of this work in other works.
A Dataset of Scratch Programs:
Scraped, Shaped and Scored
Efthimia Aivaloglou, Felienne Hermans, Jes´
us Moreno-Le´
on, and Gregorio Robles
Delft University of Technology, The Netherlands
Email: {e.aivaloglou, f.f.j.hermans}@tudelft.nl
Programamos.es & Universidad Rey Juan Carlos, Spain
Email: jesus.moreno@programamos.es
Universidad Rey Juan Carlos, Fuenlabrada (Madrid), Spain
Email: grex@gsyc.urjc.es
Abstract—Scratch is increasingly popular, both as an in-
troductory programming language and as a research target
in the computing education research field. In this paper, we
present a dataset of 250K recent Scratch projects from 100K
different authors scraped from the Scratch project repository. We
processed the projects’ source code and metadata to encode them
into a database that facilitates querying and further analysis. We
further evaluated the projects in terms of programming skills and
mastery, and included the project scoring results. The dataset
enables the analysis of the source code of Scratch projects, of their
quality characteristics, and of the programming skills that their
authors exhibit. The dataset can be used for empirical research
in software engineering and computing education.
I. INTRODUCTION
Scratch [1] is a block-based programming language de-
veloped to serve as a stepping stone for children from 8
to 16 years old to the more advanced world of computer
programming. It offers a web-based programming environment
that enables creating games and interactive animations. The
public repository of Scratch programs contains over 19 million
projects and 16 million users.
A number of works in the computing education research
field attempt to assess the programming skills that novice
programmers develop in the Scratch environment. Some utilize
program data collected during specific programming courses
(e.g., [2], [3], [4]), while others utilize the dataset made avail-
able by the Lifelong Kindergarten Group at the MIT Media
Lab [5], which contains data for Scratch projects created until
2012 (e.g., [6], [7], [8]). In addition to identifying indications
of learning of programming concepts, static analysis of Scratch
programs has also been performed for identifying code smells
and bad programming practices [9], [10], and automated qual-
ity assessment tools have been proposed (e.g., Hairball [11]
and Dr. Scratch [12]).
While Scratch is receiving increasing interest as an intro-
ductory programming language, there is no recent dataset of
Scratch programs available to the research community. The
one made available from the MIT Media Lab concerns projects
created using the previous, initial version of the Scratch
application, before the introduction of the current Scratch
version (Scratch 2) and of the web programming interface in
2013. It is since then that the popularity of Scratch started
to increase.1Scratch 2 introduced several features relating
to important programming concepts,2like custom blocks (the
equivalent to procedure definitions), and therefore that dataset
does not include projects utilizing those features.
The goal of this paper is to present an open and timely
dataset of recent Scratch programs, along with their metadata,
that can facilitate quantitative research in the fields of source
code analysis and computing education. The dataset contains
250,000 Scratch projects, from more than 100,000 different
users, that were scraped from the Scratch project repository.
It is made available as a database3which includes, for each
Scratch project, its metadata and the program data, along with
programming mastery scoring results from the Dr. Scratch
quality assessment tool [12].
II. DATASE T CONSTRUCTION
A. Data Collection
To collect the data from the web interface of the Scratch
project repository, we built a scraping program. The web
scraping program starts by reading the Scratch projects page4
and thus obtains the project identifiers of projects that were
most recently shared. Subsequently, it retrieves a JSON file
for each of the listed projects5.
We ran the scraper on March 2nd 2016 for 24 hours
and, during that time, it obtained the JSON files for
250,163 projects. Out of those, we failed to parse and further
analyze 2,367 projects due to technical difficulties with the
JSON files. The scraper and all project files are available3.
Once we obtained the Scratch projects, we parsed the JSON
files according to the specification of the format6. This resulted
in a list of used blocks per project, within the sprites and
the stage of the project. We also cross referenced all blocks
1Monthly activity trends can be found at https://scratch.mit.edu/statistics/
2The complete list of features introduced in Scratch 2 can be found at
https://wiki.scratch.mit.edu/wiki/Scratch 2.0
3https://github.com/TUDelftScratchLab/ScratchDataset
4https://scratch.mit.edu/explore/projects/all/
5For a given project id x, the program’s JSON representation can be
obtained via https://cdn.projects.scratch.mit.edu/internalapi/project/x/get
6http://wiki.scratch.mit.edu/wiki/Scratch File Format (2.0)
Aivaloglou et al. A Dataset of Scratch Programs: Scraped, Shaped and Scored
TUD-SERG-2017-007 1
with the Scratch wiki to determine their shapes and category.
For example, When Green Flag Clicked is a Hat block
from the Events category. We included blocks from Scratch
extensions, such as the LEGO WeDo extensions, that were
found in the dataset.
B. Calculation of Programming Mastery Scores
We analyzed the projects with Dr. Scratch, a tool that stati-
cally inspects Scratch projects’ source code to assign scores on
seven dimensions of computational thinking: abstraction and
problem decomposition, logical thinking, synchronization, par-
allelism, algorithmic notions of flow control, user interactivity
and data representation. These dimensions are given a value
from 0 to 3. The mastery score is the aggregated punctuation
of the seven dimensions, and therefore ranges from 0 to 21.
Dr. Scratch also detects: intra-project software cloning, i.e.,
repetition of code; the use of custom blocks, which is the way
functionality can be reused in Scratch programs; and the use
of instances of sprites, a feature labeled in Scratch as clone
creation. These values are included in the dataset.
Out of the 250,163 projects, 231,050 were successfully
analyzed with Dr. Scratch. This was due to problems with non-
official extension blocks and with some non-ASCII characters
in sprites names.
C. Importing the Data
All scraped project data and metadata, including the list
of used blocks and parameters, were imported in a relational
database. We also imported the data on the shapes and the
categories of the Scratch blocks. We then used SQL queries
for normalizing the data and bringing it in its final schema,
outlined in Table I.
III. DATASET DESCRIPTION
A. Data Representation
The projects’ data are stored in a relational database. Each
of the projects is identified by its Scratch project ID,
stored in field p_id, while its author is identified by the
username7. If a project is a remix of another one, the original
project can be found in the remixes table. Table grades
stores the Dr. Scratch results for the programming mastery
metrics per project.
The schema of the database, outlined in Table I, reflects
the structure of the Scratch programs. Projects contain sprites,
which are entities with their own associated code. The code is
organized into scripts, i.e., groups of Scratch code blocks,
each script belonging to a sprite named sprite-name. In
the example in Figure 1 there are 3 scripts, one initiated when
the green flag is clicked, one then the sprite is clicked, and
the ‘backflip’ custom block definition. Those custom blocks
are the equivalent of procedure definitions, with their names
and arguments stored in the procedures table.
7Each project and author page can be accessed in the Scratch web
interface at https://scratch.mit.edu/projects/hp idiand https://scratch.mit.edu/
users/husernameirespectively
Table Key Attribute(Description)
projects PK p_id (Scratch project ID)
project-name (name given to project)
username (author’s Scratch username)
total-views (project views number)
total-remixes (project remixes number)
total-favorites (total users favoriting)
total-loves (users ‘loving’ the project)
is-remix (calculated column, true if project
is a remix of another one)
scripts PK script_id (auto increment)
FK project_id (*...1 projects:p_id)
sprite-type ([sprite|stage|procD ef ])
sprite-name (name of sprite the script is in)
script-rank (project-level ranking of
script)
coordinates (x-y location of script in
Scratch editor)
total-blocks (calculated column, number
of blocks comprising the script)
procedures PK,FK script_id (1...1 scripts:script_id)
proc-name (name given to custom block)
total-args (number of input arguments)
blocks PK,FK script_id (*...1 scripts:script_id)
PK block-rank (script-level ranking of block)
block-type (*...1 blockTypes:b-type)
parameter-1 (value of 1st input parameter)
...
parameter-24 (value 24th input parameter)
blockTypes PK b-type (Scratch name of predefined block)
category (Scratch block category)
shape (One of 8 block shapes in Scratch)
is-input (true if block receives user input)
remixes PK remix_id (project_id of created project)
from-project_id (project_id of origi-
nal project)
grades PK,FK project_id (1...1 projects:p_id)
Abstraction (Score)
Parallelism (Score)
Logic (Score)
Synchronization (Score)
FlowControl (Score)
UserInteractivity (Score)
DataRepresentation (Score)
Mastery (Dr. Scratch total mastery score)
Clones (number of software clones)
CustomBlocks (number of user defined
blocks)
InstancesSprites (true if project uses
Scratch clones)
TABLE I
DATABAS E SC HEM A: TABLES AND ATT RI BUT ES
Fig. 1. Example of a Scratch program with three scripts in the same spite
Aivaloglou et al. A Dataset of Scratch Programs: Scraped, Shaped and Scored
2 TUD-SERG-2017-007
mean min Q1 median Q3 max
Projects per username 2.3 1 1 1 2 868
Sprites pp 5.3 0 1 2 5 525
Scripts pp 16.2 0 2 4 11 3,038
Blocks pp 144.3 0 9 26 69 34,622
Custom block def. pp 0.6 0 0 0 0 372
Total views pp 5.8 0 1 1 4 27,993
Total remixes pp 0.1 0 0 0 0 306
Total favorites pp 0.5 0 0 0 0 2,582
Dr.Scratch mastery pp 8.9 0 6 8 11 21
TABLE II
SUM MARY S TATIS TI CS O F TH E DATASE T (pp STAN DS F OR P ER P ROJ EC T).
Projects 250,163
Non-empty projects 233,491
Projects analyzed by Dr. Scratch 231,050
Remixes of other projects 12,167
Usernames 109,960
Scripts 4,049,356
Blocks 36,085,654
Custom block definitions 204,018
TABLE III
DATASET C ON TE NT S
The coding elements in the Scratch environment are the
blocks, which can receive input parameters, like block ‘go
to x:0 y:0’ in the figure. Custom clock invocations are stored
in this table just like regular blocks. To cater for the varying
number of input parameters that those blocks can have, this
table can store up to the maximum number of input parameters
found in the dataset, which is 24. The ordering of the blocks
in the scripts can be retrieved using the block-rank field,
which represents a depth-first ranking of the blocks. In the
script of the ‘backflip’ custom block definition in the example,
the ranking of the blocks would be ‘define backflip’, ‘set
counter to’, ‘+’, ‘counter’, followed by the rest of the blocks.
The clocks can be of different shapes and categories. In table
blockTypes we have stored the encoding of all types of
Scratch blocks and of the blocks from Scratch extensions.
B. Dataset Contents
As shown in Table III, the dataset contains the metadata of
250,163 Scratch projects, and the code of 233,491 of those that
are non-empty. The projects were created by 109,960 different
users. Most of the users, as shown in Table II, have created a
single project. However, 40,000 users have created more than
one and up to 868 projects in the dataset.
The majority of the projects have been viewed only once,
and they have not been favorited or remixed. Around 10,000 of
the total projects have been remixed at least once, while there
are cases of very popular projects with more than 100 remixes
and even one thousand cases with more than 100 views.
Table II also reveals the median project in the dataset:
it contains 2 sprites, 4 scripts, 26 blocks, no custom block
definitions, and receives a mastery score of 8 our of 21, which
is considered a medium level of computational thinking skills
development. The table highlights the existence of surprisingly
complex projects, with 525 sprites and over 34,000 blocks.
IV. ENA BL ED RE SEA RC H AND DATASET EXTENSION
The evaluation of block-based languages in general, and
Scratch in particular, as tools for programming education is
receiving significant research attention. A number of studies
have been carried out during the last years on understanding
the programming practices of learners in those environments,
on the programming skills they develop, and on the quality of
their programs.
The first research direction that this dataset can be utilized
for concerns the assessment of the programming skills that
novice programmers develop in the Scratch environment. The
dataset can be used for examining indications of learning of
programming concepts and abstractions, as in [10]. Existing
works in this research direction include the work by Maloney
et al. [4], who analyzed 536 Scratch projects created in an
after-school clubhouse for blocks that relate to programming
concepts and found that within the least utilized ones are
Boolean operators and variables. Another study on the in-
ternalization of programming concepts with Scratch with 46
students was presented in [2], where it was found that students
had problems with initialization, variables and concurrency.
Towards this research direction, the dataset makes the source
code of the Scratch programs available for static analysis and
can be used for quantitatively evaluating the application of
programming concepts and abstractions through the presence
of Scratch blocks of the corresponding types.
Another promising research direction relates to the quality
of the programming artifacts developed in the Scratch envi-
ronment. The dataset has already been used for quality assess-
ment, for the identification of programming smells [10], and
for exploring indications of harmful programming habits [13].
Existing works in this direction include a study in a classroom
setting [9], which highlighted two bad programming habits in
Scratch, namely bottom-up development and extremely fine-
grained programming. The authors connected the later to the
reduced use of if-blocks and finite loops and the increased
use of infinite loops. Related to smell detection are the Scratch
automated quality analysis tools Hairball [11], a static analysis
tool that can detect initialization problems and unmatched
broadcast and receive blocks, and Dr. Scratch [14], which
extends Hairball to detect two bad programming habits: not
changing the default object names and duplicating scripts.
This dataset will enable examining code quality and code
smells on a large set of Scratch projects. Static analysis of
the source code can also be performed for the identification
of other types of smells [15] that might be common in the
artifacts of novice programmers.
Moving from the software engineering to the computing
education research field, a research direction with increased
data requirements concerns the learning progressions of novice
programmers. This includes examining the factors that support
learning and improving their programming skills. Existing
works in this area have used the dataset of the previous version
of Scratch programs, created until 2012, and described in the
Introduction. They include the works of Dasgupta et al., who
Aivaloglou et al. A Dataset of Scratch Programs: Scraped, Shaped and Scored
TUD-SERG-2017-007 3
investigated how project remixing relates to the adoption of
new computational thinking concepts [8], and Yang et al., who
examined the learning patterns of programmers in terms of
block use over their first 50 projects [7].
Research towards this direction requires extending the
dataset with the complete set of projects for the included users,
along with their creation dates. This would enable examining
how their programming skills and mastery evolved through
time, possibly through remixing the projects of others. An even
richer extension of the dataset would be with periodical snap-
shots of the included projects that are still in development. This
would enable reconstructing project versioning information,
not available from the Scratch interface, and thus examining
how the projects are developed and evolve over time.
Apart from quantitative studies, the dataset can support the
design of experiments and field studies, whose material often
includes Scratch programs with specific characteristics. For
example, in [16] examples of Scratch programs exhibiting spe-
cific smells were used in a controlled experiment to determine
how the smells affect the understanding and the modification
of the programs. This queryable dataset can support finding
example programs with characteristics according to the exper-
iment design and the field of the study.
V. LIMITATIONS
The only information about the authors of the Scratch
projects contained in the dataset is their username. This
suffices for creating project portfolios of the authors and for
facilitating research in the directions described in the previous
section. However, other directions in the computing education
research field require richer author data, and especially data
on their gender and age. There are, for example, indications
that specific programming concepts are better understood
after certain ages —Seiter and Foreman [17] analyzed 150
Scratch projects from primary school students and found that
design patterns requiring the understanding of parallelism,
conditionals and, especially, variables were under-represented
by all grades apart from 5 an 6. The effect of gender and
account age were also examined in [6] in relation to the use of
programming concepts. However, this dataset does not include
information on the gender and the age of authors and cannot
be extended to do so using the current Scratch web interface
because this information is not available in the user profile
page7, even though it is provided by users upon registration.
Another limitation of this dataset is that we did not scrape
a random sample of Scratch projects, but the most recent ones
during the time that we run the scraper. It could be the case
that the programming habits of Scratch users are changing
over time. However, we counterbalanced that by collecting a
large dataset which comprises around 1.3% of all 19 million
shared Scratch projects, and is the most recent one to be made
available to the research community. Projects in the dataset are
the ones that their authors shared publicly when the scraper
run. The dataset contains cases of projects that are no longer
publicly shared and are thus no longer accessible through the
Scratch web interface.
VI. CONCLUSION
We presented a dataset of recent Scratch programs scraped
from the Scratch project repository. It is made available as
a database3which includes the source code of the Scratch
projects, their metadata, and their programming mastery scor-
ing results. The dataset can facilitate research in software
engineering and computing education, for topics like the
assessment of the programming skills that novice programmers
develop, the exploration of their learning progressions, and
issues related to the quality of the programs, such as the
identification of smells and bad programming habits.
REFERENCES
[1] M. Resnick, J. Maloney, A. Monroy-Hern´
andez, N. Rusk, E. Eastmond,
K. Brennan, A. Millner, E. Rosenbaum, J. Silver, B. Silverman, and
Y. Kafai, “Scratch: Programming for All, Commun. ACM, vol. 52,
no. 11, pp. 60–67, Nov. 2009.
[2] O. Meerbaum-Salant, M. Armoni, and M. M. Ben-Ari, “Learning
Computer Science Concepts with Scratch,” in Proceedings of ICER
2010. New York, NY, USA: ACM, 2010, pp. 69–76.
[3] A. Wilson, T. Hainey, and T. Connolly, “Evaluation of computer
games developed by primary school children to gauge understanding
of programming concepts,” in European Conference on Games Based
Learning. Academic Conferences International Limited, 2012, p. 549.
[4] J. H. Maloney, K. Peppler, Y. Kafai, M. Resnick, and N. Rusk, “Pro-
gramming by choice: Urban youth learning programming with Scratch,”
in Proceedings of the 39th SIGCSE. ACM, 2008, pp. 367–371.
[5] B. M. Hill and A. Monroy-Hern´
andez, “A longitudinal dataset of ve
years of public activity in the Scratch online community, Scientific
Data, vol. 4, pp. 170 002 EP –, 01 2017.
[6] D. A. Fields, M. Giang, and Y. Kafai, “Programming in the wild: Trends
in youth computational participation in the online Scratch community,
in Proceedings of the 9th WiPSCE. ACM, 2014, pp. 2–11.
[7] S. Yang, C. Domeniconi, M. Revelle, M. Sweeney, B. U. Gelman,
C. Beckley, and A. Johri, “Uncovering trajectories of informal learning
in large online communities of creators,” in Proceedings of the Second
ACM Conference on Learning @ Scale. ACM, 2015, pp. 131–140.
[8] S. Dasgupta, W. Hale, A. Monroy-Hern´
andez, and B. M. Hill, “Remixing
as a pathway to Computational Thinking,” in Proceedings of the 19th
ACM Conference on Computer-Supported Cooperative Work & Social
Computing, ser. CSCW ’16. ACM, 2016, pp. 1438–1449.
[9] O. Meerbaum-Salant, M. Armoni, and M. Ben-Ari, “Habits of Program-
ming in Scratch,” in Proceedings of the 16th Annual Joint Conference on
Innovation and Technology in Computer Science Education, ser. ITiCSE
’11. New York, NY, USA: ACM, 2011, pp. 168–172.
[10] E. Aivaloglou and F. Hermans, “How kids code and how we know: An
exploratory study on the Scratch repository, in Proceedings of ICER
2016. ACM, 2016, pp. 53–61.
[11] B. Boe, C. Hill, M. Len, G. Dreschler, P. Conrad, and D. Franklin, “Hair-
ball: Lint-inspired Static Analysis of Scratch Projects,” in Proceeding of
the 44th SIGCSE. New York, NY, USA: ACM, 2013, pp. 215–220.
[12] J. Moreno and G. Robles, Automatic detection of bad programming
habits in Scratch: A preliminary study, in 2014 IEEE Frontiers in
Education Conference (FIE), Oct. 2014, pp. 1–4.
[13] G. Robles, J. Moreno-Le´
on, E. Aivaloglou, and F. Hermans, “Software
clones in Scratch projects: On the presence of copy-and-paste in Com-
putational Thinking learning,” in Proceedings of the 11th International
Workshop on Software Clones, 2017, pp. 31–37.
[14] J. Moreno-Le´
on, G. Robles, and M. Rom´
an-Gonz´
alez, “Dr. Scratch:
Automatic Analysis of Scratch Projects to Assess and Foster
Computational Thinking,” RED : Revista de Educaci´
on a Distancia,
no. 46, pp. 1–23, Jan. 2015. [Online]. Available: https://doaj.org
[15] M. Fowler, Refactoring: Improving the design of existing code. Boston,
MA, USA: Addison-Wesley Longman Publishing Co., Inc., 1999.
[16] F. Hermans and E. Aivaloglou, “Do code smells hamper novice program-
ming? A controlled experiment on Scratch programs,” in 2016 IEEE 24th
International Conference on Program Comprehension, 2016, pp. 1–10.
[17] L. Seiter and B. Foreman, “Modeling the learning progressions of
Computational Thinking of primary grade students,” in Proceedings of
ICER 2013. ACM, 2013, pp. 59–66.
Aivaloglou et al. A Dataset of Scratch Programs: Scraped, Shaped and Scored
4 TUD-SERG-2017-007
TUD-SERG-2017-007
ISSN 1872-5392
... Other studies have analyzed the CT skills of beginner programmers developed in the Scratch environment owing to Scratch projects and CT skills being correlated [15][16][17][18]. Aivaloglou et al. presented an open database of 250,163 scraped Scratch projects to facilitate quantitative research in source code analysis and computing education [19]. Their data size was statistically sound; however, it had a limitation when creating a project portfolio since the only demographic information in the dataset was the username. ...
... In addition, previous research faced challenges due to limited sample sizes and a lack of user information. Although the user portfolio of Oluk and Korkmaz [14] has richer information than that of Aivaloglou et al. [19], the sample size was still limited. Hence, the outcome may be specific to the area where the research was performed, and so the existing dataset inhibits drawing a broader conclusion. ...
... Thus, a more effective approach is required. Further, the sample size in Aivaloglou et al. [19] seems promising, but the number of nominal variables is insufficient for hypothetical testing. ...
Article
Full-text available
The Computer Science industry suffers from a vivid gender gap. To understand this gap, Computational Thinking skills in Computer Science education are analyzed by binary gender roles using block-based programming languages such as Scratch since they are intuitive for beginners. Platforms such as Dr. Scratch, aid learners in improving their coding skills by earning a Computational Thinking score while supporting effective assessments of students' projects and fostering basic computer programming. Although previous studies have examined gender differences using Scratch programs, few have analyzed the Scratch project type's impact on the evaluation process when comparing genders. Herein, the influence of project type is analyzed using instances of 124 (62 male, 62 female) projects on the Scratch website. Initially, projects were categorized based on the user's gender and project type. Hypothetical testing of each case shows that the scoring system has a bias based on the project type. As gender differences appear by project type, the project type may significantly affect the gender gap in Computational Thinking scores. This study demonstrates the importance of incorporating the project type's effect into the Scratch projects' evaluation process when assessing gender differences.
... However, it is notable that the most complex project was implemented by a female pair (ICC = 62, Figure 8 no significant differences (p = 0.240). Overall, the projects are comparatively complex for an introductory programming project, which we explicitly encouraged through the course design (e.g., game rather than story) in order to improve the girls' adaptation to programming concepts [44], [53], [59]. 5) Creativity: SCRATCH enables young students to express their creativity while programming, which supports learning and motivational success [57]. ...
... However, it is notable that the most complex project was implemented by a female pair (ICC = 62, Figure 8 no significant differences (p = 0.240). Overall, the projects are comparatively complex for an introductory programming project, which we explicitly encouraged through the course design (e.g., game rather than story) in order to improve the girls' adaptation to programming concepts [44], [53], [59]. 5) Creativity: SCRATCH enables young students to express their creativity while programming, which supports learning and motivational success [57]. ...
Preprint
Full-text available
Young learners are increasingly introduced to programming , and one of the main challenges for educators is to achieve learning success while also creating enthusiasm. As it is particularly difficult to achieve this enthusiasm initially in young females, prior work has identified gender-specific differences in the programming behavior of young learners. Since pair programming, which turns programming into a more sociable activity , has been proposed as an approach to support programming education, in this paper we aim to investigate whether similar gender-specific characteristics can also be observed during pair programming. Therefore, we designed a gender-neutral introductory SCRATCH programming course tailored for integrating pair programming principles, and conducted it with a total of 139 students aged between 8 and 14 years. To identify gender-dependent differences and similarities, we measure the attitude towards programming and the course setting, observe the behavior of the students while programming, and analyze the code of the programs for different gender-combinations. Overall, our study demonstrates that pair programming is well suited for young learners and results in a positive attitude. While the resulting programs are similar in quality and complexity independent of gender, differences are evident when it comes to the compliance to pair programming roles, the exploration of code, and the creative customization of programs. These findings contribute to an in-depth understanding of social and technical gender specifics of pair programming, and provide educators with resources and guidance for implementing gender-sensitive pair programming in the classroom.
... Hill and Monroy-Hernández [19] created a large dataset of Scratch projects over 5 years although they did not perform any analysis on the dataset themselves. In simultaneous work, Aivaloglou et al. [3] created a dataset of 250,000 Scratch projects but did not analyse it temporally. ...
Article
Objectives Java is a popular programming language for use in computing education, but it is difficult to get a wide picture of the issues that it presents for novices, and most studies look only at the types or frequency of errors. In this observational study we aim to learn how novices use different features of the Java language. Participants Users of the BlueJ development environment have been invited to opt-in to anonymously record their activity data for the past eight years. This dataset is called Blackbox, which was used as the basis for this study. BlueJ users are mostly novice programmers, predominantly male, with a median age of 16. Our data subset featured approximately 225,000 participants from around the world. Study Methods We performed a secondary data analysis that used data from the Blackbox dataset. We examined over 320,000 Java projects collected over the course of eight years, and used source code analysis to investigate the prevalence of various specifically-selected Java programming usage patterns. As this was an observational study without specific hypotheses, we did not use significance tests; instead we present the results themselves with commentary, having applied seasonal trend decomposition to the data. Findings We found many long-term trends in the data over the course of the eight years, most of which were monotonic. There was a notable reduction in the use of the main method (common in Java but unnecessary in BlueJ), and a general reduction in the complexity of the projects. We find that there are only a small number of frequently used types: int, String, double and boolean, but also a wide range of other infrequently used types. Conclusions We find that programming usage patterns gradually change over a long period of time (a period where the Java language was not seeing major changes), once seasonal patterns are accounted for. Any changes are likely driven by instructors and the changing demographics of programming novices. The novices use a relatively restricted subset of Java, which implies that designers of languages specifically targeted at novices can satisfy their needs with a smaller set of language constructs and features. We provide detailed recommendations for the designers of educational programming languages and supporting development tools.
Research
Full-text available
This study aimed to investigate the effect of web-based collaborative learning method with Scratch software on the critical thinking skills of 5th grade students and to create materials in Scratch software on the biodiversity subject. This study, in which a quasi-experimental design with pre-test-post-test control group was used, was carried out with 96 fifth grade students selected using convenient sampling method. In the implementation of the study, in addition to the education and training activities on biodiversity in the control groups, the students in the experimental group created products using the web-based collaborative learning method with Scratch in science and information technologies and software courses. As a result of the t-test for the independent samples of the post-test data obtained from the critical thinking questionnaire, it was determined that the critical thinking skills of the experimental group students were higher than the control group students. It is recommended for younger individuals to use Scratch to come up with solutions by using critical thinking skills to cope with complex problems affecting the whole world such as global warming, wars, and pandemics, and to choose the right solution amongst the given ones. Teachers can get an idea about students' opinions, conceptual errors, and their capability to express themselves, and they can reorganize the educational activities by taking these variables into consideration.
Article
Full-text available
Many software engineering research papers rely on time-based data (e.g., commit timestamps, issue report creation/update/close dates, release dates). Like most real-world data however, time-based data is often dirty. To date, there are no studies that quantify how frequently such data is used by the software engineering research community, or investigate sources of and quantify how often such data is dirty. Depending on the research task and method used, including such dirty data could affect the research results. This paper presents an extended survey of papers that utilize time-based data, published in the Mining Software Repositories (MSR) conference series. Out of the 754 technical track and data papers published in MSR 2004–2021, we saw at least 290 (38%) papers utilized time-based data. We also observed that most time-based data used in research papers comes in the form of Git commits, often from GitHub. Based on those results, we then used the Boa and Software Heritage infrastructures to help identify and quantify several sources of dirty Git timestamp data. Finally we provide guidelines/best practices for researchers utilizing time-based data from Git repositories.
Conference Paper
Full-text available
Computer programming is being introduced in schools worldwide as part of a movement that promotes Computational Thinking (CT) skills among young learners. In general, learners use visual, block-based programming languages to acquire these skills, with Scratch being one of the most popular ones. Similar to professional developers, learners also copy and paste their code, resulting in duplication. In this paper we present the findings of correlating the assessment of the CT skills of learners with the presence of software clones in over 230,000 projects obtained from the Scratch platform. Specifically, we investigate i) if software cloning is an extended practice in Scratch projects, ii) if the presence of code cloning is independent of the programming mastery of learners, iii) if code cloning can be found more frequently in Scratch projects that require specific skills (as parallelism or logical thinking), and iv) if learners who have the skills to avoid software cloning really do so. The results show that i) software cloning can be commonly found in Scratch projects, that ii) it becomes more frequent as learners work on projects that require advanced skills, that iii) no CT dimension is to be found more related to the absence of software clones than others, and iv) that learners -even if they potentially know how to avoid cloning- still copy and paste frequently. The insights from this paper could be used by educators and learners to determine when it is pedagogically more effective to address software cloning, by educational programming platform developers to adapt their systems, and by learning assessment tools to provide better evaluations.
Article
Full-text available
Scratch is a programming environment and an online community where young people can create, share, learn, and communicate. In collaboration with the Scratch Team at MIT, we created a longitudinal dataset of public activity in the Scratch online community during its first five years (2007–2012). The dataset comprises 32 tables with information on more than 1 million Scratch users, nearly 2 million Scratch projects, more than 10 million comments, more than 30 million visits to Scratch projects, and more. To help researchers understand this dataset, and to establish the validity of the data, we also include the source code of every version of the software that operated the website, as well as the software used to generate this dataset. We believe this is the largest and most comprehensive downloadable dataset of youth programming artifacts and communication.
Conference Paper
Full-text available
Block-based programming languages like Scratch, Alice and Blockly are becoming increasingly common as introductory languages in programming education. There is substantial research showing that these visual programming environments are suitable for teaching programming concepts. But, what do people do when they use Scratch? In this paper we explore the characteristics of Scratch programs. To this end we have scraped the Scratch public repository and retrieved 250,000 projects. We present an analysis of these projects in three different dimensions. Initially, we look at the types of blocks used and the size of the projects. We then investigate complexity, used abstractions and programming concepts. Finally we detect code smells such as large scripts, dead code and duplicated code blocks. Our results show that 1) most Scratch programs are small, however Scratch programs consisting of over 100 sprites exist, 2) programming abstraction concepts like procedures are not commonly used and 3) Scratch programs do suffer from code smells including large scripts and unmatched broadcast signals.
Conference Paper
Full-text available
Most research in primary and secondary computing education has focused on understanding learners within formal classroom communities, leaving aside the growing number of promising informal online programming communities where young learners contribute, comment, and collaborate on programs. In this paper, we examined trends in computational participation in Scratch, an online community with over 1 million registered youth designers primarily 11-18 years of age. Drawing on a random sample of 5,000 youth programmers and their activities over three months in early 2012, we examined the quantity of programming concepts used in projects in relation to level of participation, gender, and account age of Scratch programmers. Latent class analyses revealed four unique groups of programmers. While there was no significant link between level of online participation, ranging from low to high, and level of programming sophistication, the exception was a small group of highly engaged users who were most likely to use more complex programming concepts. Groups who only used few of the more sophisticated programming concepts, such as Booleans, variables and operators, were identified as Scratch users new to the site and girls. In the discussion we address the challenges of analyzing young learners' programming in informal online communities and opportunities for designing more equitable computational participation.
Conference Paper
Full-text available
Using the Scratch environment as a tool to teach programming skills or develop computational thinking is increasingly common in all levels of education, well-documented case studies from primary school to university can be found. However, there are reports that indicate that students learning to program in this environment show certain habits that are contrary to the basic programming recommendations. In our work as instructors with high school students, we have detected these and other bad practices, such as the repetition of code and object naming, on a regular basis. This paper focuses on verifying whether these issues can generally be found in the Scratch community, by analyzing a large number of projects available on the Scratch community website. To test this hypothesis, we downloaded 100 projects and analyzed them with two plug-ins we developed to automatically detect these bad practices. The plug-ins extend the functionality of the Hairball tool, a static code analyzer for Scratch projects. The results obtained show that, in general, projects in the repository also incur in the investigated malpractices. Along with suggestions for future work, some ideas that might assist to address such situations are proposed in the conclusions of the paper.
Article
Full-text available
Using the Scratch environment as a tool to teach programming skills or develop computational thinking is increasingly common in all levels of education, well-documented case studies from primary school to university can be found. However, there are reports that indicate that students learning to program in this environment show certain habits that are contrary to the basic programming recommendations. In our work as instructors with high school students, we have detected these and other bad practices, such as the repetition of code and object naming, on a regular basis. This paper focuses on verifying whether these issues can generally be found in the Scratch community, by analyzing a large number of projects available on the Scratch community website. To test this hypothesis, we downloaded 100 projects and analyzed them with two plug-ins we developed to automatically detect these bad practices. The plug-ins extend the functionality of the Hairball tool, a static code analyzer for Scratch projects. The results obtained show that, in general, projects in the repository also incur in the investigated malpractices. Along with suggestions for future work, some ideas that might assist to address such situations are proposed in the conclusions of the paper.
Article
Theorists and advocates of "remixing" have suggested that appropriation can act as a pathway for learning. We test this theory quantitatively using data from more than 2.4 million multimedia programming projects shared by more than 1 million users in the Scratch online community. First, we show that users who remix more often have larger repertoires of programming commands even after controlling for the numbers of projects and amount of code shared. Second, we show that exposure to computational thinking concepts through remixing is associated with increased likelihood of using those concepts. Our results support theories that young people learn through remixing, and have important implications for designers of social computing systems.
Article
Scratch is a visual programming environment that is widely used by young people. We investigated if Scratch can be used to teach concepts of computer science (CS). We developed learning materials for middle-school students that were designed according to the constructionist philosophy of Scratch and evaluated them in a few schools during two years. Tests were constructed based upon a novel combination of the revised Bloom taxonomy and the Structure of the Observed Learning Scratch is a visual programming environment that is widely used by young people. We investigated if Scratch can be used to teach concepts of computer science (CS). We developed learning materials for middle-school students that were designed according to the constructionist philosophy of Scratch and evaluated them in a few schools during two years. Tests were constructed based upon a novel combination of the revised Bloom taxonomy and the Structure of the Observed Learning Outcome taxonomy. These instruments were augmented with qualitative tools, such as observations and interviews. The results showed that students could successfully learn important concepts of CS, although there were problems with some concepts such as repeated execution, variables, and concurrency. We believe that these problems can be overcome by modifications to the teaching process that we suggest. Outcome taxonomy. These instruments were augmented with qualitative tools, such as observations and interviews. The results showed that students could successfully learn important concepts of CS, although there were problems with some concepts such as repeated execution, variables, and concurrency. We believe that these problems can be overcome by modifications to the teaching process that we suggest.
Article
Under the Curriculum for Excellence (CfE) in Scotland, newer approaches such as games-based learning and games-based construction are being adopted to motivate and engage students. Construction of computer games is seen by some to be a highly motivational and practical approach at engaging children at Primary Education (PE) level in computer programming concepts. Games-based learning (GBL) and gamesbased construction both suffer from a dearth of empirical evidence supporting their validity as teaching and learning approaches. To address this issue, this paper will present the findings of observational research at PE level using Scratch as a tool to develop computer games using rudimentary programming concepts. A list of criteria will be compiled for reviewing the implementation of each participant to gauge the level of programming proficiency demonstrated. The study will review 29 games from Primary 4 to Primary 7 level and will present the overall results and results for each individual year. This study will contribute to the empirical evidence in gamesbased construction by providing the results of observational research across different levels of PE and will provide pedagogical guidelines for assessing programming ability using a games-based construction approach.