Content uploaded by Kyriakos Tsoukalas
Author content
All content in this area was uploaded by Kyriakos Tsoukalas on Sep 28, 2021
Content may be subject to copyright.
On Aective States in Computational Cognitive Practice
through Visual & Musical Modalities
Kyriakos Tsoukalas
Dissertation submitted to the Faculty of the
Virginia Polytechnic Institute and State University
in partial fulllment of the requirements for the degree of
Doctor of Philosophy
Ivica I. Bukvic, Chair
Kevin D. Carlson
Adrienne Holz Ivory
R. Benjamin Knapp
May 24, 2021
Blacksburg, Virginia
Keywords: Aective States, Musical Computing, Multimodal Interaction, Computational Thinking
Copyright © 2021 Kyriakos Tsoukalas
On Aective States in Computational Cognitive Practice
through Visual & Musical Modalities
Kyriakos Tsoukalas
Academic Abstract
Learners’ aective states correlate with learning outcomes. A key aspect of instructional
design is the choice of modalities by which learners interact with instructional content. The
existing literature focuses on quantifying learning outcomes without quantifying learners’
aective states during instructional activities. An investigation of how learners feel during
instructional activities will inform the instructional systems design methodology of a method
for quantifying the eects of individually available modalities on learners’ aect.
The objective of this dissertation is to investigate the relationship between aective states
and learning modalities of instructional computing. During an instructional activity, learn-
ers’ enjoyment, excitement, and motivation are measured before and after a computing ac-
tivity oered in three distinct modalities. The modalities concentrate on visual and musical
computing for the practice of computational thinking. An aective model for the practice
of computational thinking through musical expression was developed and validated.
This dissertation begins with a literature review of relevant theories on embodied cogni-
tion, learning, and aective states. It continues with designing and fabricating a prototype
instructional apparatus and its virtual simulation as a web service, both for the practice of
computational thinking through musical expression, and concludes with a study investigat-
ing participants’ aective states before and after four distinct online computing activities.
This dissertation builds on and contributes to extant literature by validating an aec-
tive model for computational thinking practice through self-expression. It also proposes a
nomological network for the construct of computational thinking for future exploration of
the construct, and develops a method for the assessment of instructional activities based on
predened levels of skill and knowledge.
On Aective States in Computational Cognitive Practice
through Visual & Musical Modalities
Kyriakos Tsoukalas
General Audience Abstract
This dissertation investigates the role of learners’ aect during instructional activities of
visual and musical computing. More specically, learners’ enjoyment, excitement, and moti-
vation are measured before and after a computing activity oered in four distinct ways. The
computing activities are based on a prototype instructional apparatus, which was designed
and fabricated for the practice of computational thinking. A study was performed using
a virtual simulation accessible via internet browser. The study suggests that maintaining
enjoyment during instructional activities is a more direct path to academic motivation than
excitement.
Acknowledgments
I thank my advisor Dr. Ivica Ico Bukvic for his guidance and motivation during my doctoral
research.
I thank my advisory committee members, Dr. Kevin D. Carlson, Dr. Adrienne Holz Ivory,
and Dr. R. Benjamin Knapp, for their support and advice throughout my studies and
research.
I thank the Institute for Creativity, Arts, and Technology at Virginia Tech for partial support
of this study.
This work received support from the Institute for Creativity, Arts, and Technology at Virginia Polytechnic
Institute and State University.
iv
Contents
1 Introduction 1
1.1 Instructional Systems ........................................ 1
1.1.1 Instructional Computing ................................... 1
1.1.2 Self-Regulated Learning ................................... 2
1.1.3 Human-Centered Design ................................... 3
1.1.4 Virtual-Physical Manipulatives ............................... 3
1.2 Aective States ............................................ 4
1.2.1 Hedonic Tone & Pleasure-Arousal Theory ......................... 4
1.2.2 Motivational Salience .................................... 5
1.3 Self-Expression ............................................ 6
1.3.1 Communication ........................................ 6
1.3.2 Musical Expression ...................................... 7
1.4 Research Characterization ...................................... 7
1.5 Methodological Contributions .................................... 8
2 Literature Review 9
2.1 Embodied Cognition ......................................... 9
2.1.1 System-View of Embodiment ................................ 9
2.1.2 Notions of Embodiment ................................... 11
2.1.3 Embodiment in Learning Theories ............................. 13
2.2 Sense-Making ............................................. 16
2.2.1 Semiotic Resources ...................................... 16
2.2.2 Semiotic Modes ........................................ 16
2.2.3 Inter-Relations of Semiotic Resources ........................... 17
2.2.4 Aordances of Semiotic Modes ............................... 17
2.3 Computational Thinking ....................................... 18
2.3.1 Conceptual Denitions .................................... 18
2.3.2 Quantication ........................................ 20
2.3.3 Educational Applications .................................. 22
2.3.4 Gamication and Musical Expression ............................ 24
v
2.3.5 Critique ............................................ 28
2.3.6 Proposed Nomological Network ............................... 30
2.4 Systems Design ............................................ 33
2.4.1 Design Patterns for Self-Expression ............................. 33
2.4.2 Cognitive Mode ........................................ 34
2.4.3 Aective Mode ........................................ 36
2.4.4 Behavioral Mode ....................................... 37
2.4.5 Assessment of levels of Knowledge and Thinking ..................... 38
2.5 Propositions .............................................. 40
3 Research Design & Methodology 42
3.1 Introduction .............................................. 42
3.2 Design ................................................. 42
3.2.1 Operationalization of Cognitive Mode ........................... 42
3.2.2 Operationalization of Aective Mode ............................ 43
3.2.3 Operationalization of Behavioral Mode ........................... 44
3.3 The Device .............................................. 44
3.4 Protocol ................................................ 45
3.4.1 Sample ............................................ 49
3.5 Measures ............................................... 49
3.5.1 Measure of Computational Knowledge ........................... 49
3.5.2 Measure of Core-Aect (Enjoyment & Excitement) .................... 50
3.5.3 Measure of Motivational Salience .............................. 50
3.6 Hypotheses Testing .......................................... 51
3.7 Inferential Statistics ......................................... 54
4 Data Analysis 56
4.1 Descriptive Statistics ......................................... 56
4.1.1 Internal Reliability ...................................... 59
4.2 Findings ................................................ 60
4.2.1 Hypotheses 1.* ........................................ 60
4.2.2 Hypotheses 2.* ........................................ 61
vi
4.2.3 Hypotheses 3.* ........................................ 62
4.2.4 Computational Knowledge .................................. 63
4.2.5 Enjoyment & Excitement .................................. 64
4.2.6 Motivational Salience .................................... 65
4.2.7 Demographics ......................................... 67
4.2.8 Subject Pooling ........................................ 67
4.2.9 Feedback Textual Analysis ................................. 68
4.3 Summary ............................................... 72
5 Discussion 75
5.1 Future Research ........................................... 80
5.2 Conclusions .............................................. 81
6 Appendix A 83
6.1 Long Textual Tables of Literature Review ............................. 83
6.2 Protocol Details ........................................... 92
6.2.1 Quiz of Prior Computational Knowledge .......................... 98
6.2.2 Quiz of Subsequent Computational Knowledge ...................... 101
7 Appendix B 104
7.1 Numerical Tables of Statistical Analysis ..............................104
References 120
vii
List of Figures
1 A Nomological Network for Computational Thinking. ...................... 33
2 Basic Relational Trees of Dataow Networks. ........................... 35
3 Basic Trees Representing Multimodality. .............................. 36
4 An Aective Model for the Practice of Computational Thinking through Self-Expression. . . 38
5 A Self-Regulated Articial Coupling for Expression. ....................... 43
6 Web-based Computing - Visual Programming. ........................... 45
7 Web-based Computing - Programmable Virtual Musical Instrument. .............. 45
8 A Mixed Factorial Experimental Design. Descriptions of the steps are available in Appendix
A, Section 6.2. ............................................ 46
9 Aective Sliders for Self-Reported Enjoyment and Excitement [173]. .............. 50
10 Empirical Data Model: Correlations among Variables (see Appendix B, Table 47). . . . . . . 63
11 Frequent Terms in Subjects’ Comments. Larger letter size means higher number of observations. 69
12 Aect Analysis of Subjects’ Textual Comments for Treatment 1, Visual Programming. . . . . 70
13 Aect Analysis of Subjects’ Textual Comments for Treatment 2, Visual Programming for
Sound Production. .......................................... 70
14 Aect Analysis of Subjects’ Textual Comments for Treatment 3, Visual Programming for
Sound Production including a Virtual Musical Instrument. .................... 71
15 Aect Analysis of Subjects’ Textual Comments for Treatment 4, Video Watching of Oceanic
Scenery. ................................................ 71
16 Aect Analysis of Subjects’ Textual Comments for whole Sample. ................ 72
17 After Treatment Eect on Computational Knowledge and Aective States. .......... 74
18 Aective Slider: Self-Assessment of Aective Arousal (certain graphics used with creative
commons license from [173]). .................................... 92
19 Aective Slider: Self-Assessment of Aective Pleasure (certain graphics used with creative
commons license from [173]). .................................... 92
20 Treatment 1; Visual-Programming. ................................. 95
21 Treatment 2; Visual-Programming for Sound Production. .................... 96
22 Treatment 3; Visual-Programming for Sound Production including a Virtual Musical
Instrument. .............................................. 96
viii
List of Tables
1 A Procedural View of Computational Thinking. .......................... 31
2 A Diagnostic-Formative-Summative Assessment Matrix. ..................... 39
3 Generalized Diagnostic-Formative-Summative Assessment Matrix. ............... 40
4 Assessment Matrix Pertaining to the Interval Scale. ....................... 43
5Semiotic Resources: Inter-semiotic Relationships. ......................... 44
6 ANCOVA Summary. ......................................... 51
7 Repeated Measures ANOVA Summary per Treatment. ...................... 52
8 ANOVA Summary. .......................................... 53
9 Sample Demographics - Age ..................................... 56
10 Sample Demographics - Gender ................................... 57
11 Sample Descriptive Statistics .................................... 58
12 Research Session - Duration of Survey Parts ............................ 59
13 Probability Estimation - Predened Characteristics across Treatment Groups ......... 68
14 Sentiment Analysis of Subjects’ Comments ............................ 69
15 Denitions and Perspectives of Computational Thinking. ..................... 85
16 Computational Thinking Modules in Curricula for all Education Levels. ............ 90
17 Educational Tools for the Development of Computational Thinking Skills. ........... 92
18 Descriptive Statistics of Sample (duplicate of Table 11 for reference within Section 7) . . . . . 104
19 Descriptive Statistics of Subject Pool with more than .083 ∆CK .................105
20 Descriptive Statistics of Subject Pool with less than .083 ∆CK ..................105
21 Descriptive Statistics of Subject Pool with Positive ∆CK .....................106
22 Descriptive Statistics of Subject Pool with Negative ∆CK ....................106
23 Descriptive Statistics of Subject Pool with Positive ∆Enj .....................107
24 Descriptive Statistics of Subject Pool with Negative ∆Enj ....................107
25 Descriptive Statistics of Subject Pool with Positive ∆Exc .....................108
26 Descriptive Statistics of Subject Pool with Negative ∆Exc ....................108
27 Descriptive Statistics of Subject Pool with Positive ∆Enj and Negative ∆Exc .......... 109
28 Descriptive Statistics of Subject Pool with Negative ∆Enj and Positive ∆Exc .......... 109
29 Descriptive Statistics of Subject Pool with both Positive ∆Enj and ∆Exc ............110
ix
30 Descriptive Statistics of Subject Pool with both Negative ∆Enj and ∆Exc ............110
31 Descriptive Statistics of Subject Pool with Positive ∆CK and Negative ∆Enj .......... 111
32 Descriptive Statistics of Subject Pool with Negative ∆CK and Positive ∆Enj .......... 111
33 Descriptive Statistics of Subject Pool with both Positive ∆CK and ∆Enj ............112
34 Descriptive Statistics of Subject Pool with both Negative ∆CK and ∆Enj ............112
35 Descriptive Statistics of Subject Pool with Positive ∆CK and Negative ∆Exc .......... 113
36 Descriptive Statistics of Subject Pool with Negative ∆CK and Positive ∆Exc .......... 113
37 Descriptive Statistics of Subject Pool with both Positive ∆CK and ∆Exc ............114
38 Descriptive Statistics of Subject Pool with both Negative ∆CK and ∆Exc ............114
39 Descriptive Statistics of Subject Pool with Positive ∆CK, Positive ∆Enj, and ∆Exc . . . . . . 115
40 Descriptive Statistics of Subject Pool with Positive ∆CK, Positive ∆Enj, and Negative ∆Exc . 115
41 Descriptive Statistics of Subject Pool with Positive ∆CK, Negative ∆Enj, and Positive ∆Exc . 116
42 Descriptive Statistics of Subject Pool with Positive ∆CK, Negative ∆Enj, and Negative ∆Exc . 116
43 Descriptive Statistics of Subject Pool with Negative ∆CK, Positive ∆Enj, and Positive ∆Exc . 117
44 Descriptive Statistics of Subject Pool with Negative ∆CK, Positive ∆Enj, and Negative ∆Exc . 117
45 Descriptive Statistics of Subject Pool with Negative ∆CK, Negative ∆Enj, and Positive ∆Exc . 118
46 Descriptive Statistics of Subject Pool with Negative ∆CK,∆Enj, and ∆Exc ........... 118
47 Descriptive Statistics - Correlation Table of Sample Variables ..................119
x
1 Introduction
1.1 Instructional Systems
1.1.1 Instructional Computing
Instructional computing is not limited to programming; it refers mostly to instructional
activities that are achieved through the use of computers. The learning modality of an in-
structional activity has an impact on the aective states and self-ecacy of learners, both
which promote self-regulated learning [1, pp. 207-231]. A signicant aspect of this disserta-
tion is the exploration of self-expressive computing in order to increase the aective quality
of instructional activities.
This dissertation contextualizes computational thinking [2] as a cognitive practice, ex-
plores the aective quality of computing for self-expression [3], and endorses an experiential
approach to learning [4, pp. 121-130], which focuses on learners’ aect [5]. An aective model
of practicing computational thinking through self-expression is proposed and exemplied by
an instructional apparatus that enables visual programming for sound production and mu-
sical expression [6,7,8,9]. Empirical evidence is presented from a study that utilized a
virtual simulation of the physical instructional apparatus accessible via a web browser. The
study investigates the role of learners’ aect during three distinct modes of computing: vi-
sual programming, visual programming for sound production, and visual programming for
sound production including a virtual musical instrument.
The construct of computational thinking postulates that problems and their solutions
be described with semiotic resources interpretable by computers. A semiotic resource is a
coherent pattern of sensory-input regarded as a means for making meaning, such as icons,
sound, or text. Computational thinking has been dened as “thought processes” [10] that
depend on a set of skills [11,12,13,14,15,16,17,18]. There are direct (physiological)
and indirect (behavioral) ways to measure the intensity of thinking. A direct approach is
1
concerned with physiological data such as “some summary statistic of brain activation.” An
indirect approach is concerned with a “person’s score on some behavioral measure” [19, p. 7].
A prevalent indirect approach in previous research focuses on measuring the outcome from
practicing computational tasks. For example, an increase in skills that can be observed by
means of a pretest-posttest [20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37].
Previous researchers largely agree that computational thinking skills consist of, but are not
limited to, the skills of “problem-solving” [38,39,40], “code literacy” [38,40,41], “pattern
recognition” [42,43,44], and “data analysis” [40,45,46].
1.1.2 Self-Regulated Learning
Self-motivation and self-regulation are characteristics of eective learners [47,48]. Self-
regulated learning involves having some control over learning activities [49,50,51]. To
facilitate self-regulated learning, instructional systems may oer alternative learning modal-
ities for learners to choose from [52, pp. 189-205].
The theory of experiential learning focuses on dierent models of human aect [53, pp.43-
44]. Categorizing learning modalities abstracts learning styles that have been conceptual-
ized to range from a “concrete experience” to an “abstract conceptualization,” and from
“active experimentation” to “reective observation” [54, pp.227-245]. Kolb et al. point
out four ”specialized” learning styles that were observed empirically: “Diverging” (concrete
experience and reective observation), “Assimilating” (abstract conceptualization and re-
ective observation) “Converging” (abstract conceptualization and active experimentation),
and “Accommodating” (concrete experience and active experimentation) [54, pp.230-241].
Instructional systems capable of oering multiple learning modalities to accommodate dif-
ferent learning styles or a space among them, are multimodal. Choosing a learning modality
is essentially a choice among dierent semiotic modes that learners may use to make sense
of instructional activities [55].
2
1.1.3 Human-Centered Design
The human-centered approach to design is generally conceptualized in three phases: rst,
an observation of social activities related the situation of focus drives the conceptualization
of a problem; second, an iterative process for the imagination of possible solutions develops,
tests, analyzes, and improves designs; third, the most socially appropriate design solution is
chosen for implementation.
Factual knowledge about a problem does not equal the understanding of it. Therefore,
the rst phase seeks to make sense of a problem’s inner workings by observing it under
real-world conditions. After developing an understanding of the problem, designers ideate
feasible solutions, which should then be evaluated for practicality and scalability [56,57].
Designs can be developed in parallel and/or iteratively. Parallel designing creates com-
petition among designs (inter-design), and iterative designing creates competition among
design iterations (intra-design). Inter-design competition explores incompatible solutions in
parallel and gathers segmented knowledge, which can accelerate the development of design-
solutions. The main advantage of parallel design is that creativity is less restricted by
previous choices because designs can fork to enable the exploration of incompatible solu-
tions. The main disadvantage of parallel design is that it does not concentrate resources on
one design process. Imagining design-solutions involves parallel design, while implementing
design-solutions involves progressive renement through iterative design [58]. Lastly, it is
important to eciently monitor the implementation of a solution in order to prevent errors
or identify them early enough and to manage potential risks.
1.1.4 Virtual-Physical Manipulatives
Instructional manipulatives exemplify concepts and enable self-regulated experimentation.
Learners can use manipulatives to explore phenomena and learn by reecting on their obser-
vations. In their work, Weick et al. argue that sense-making “involves turning circumstances
3
into a situation that is comprehended explicitly in words and that serves as a springboard
into action”[59, p. 409]. Instructional manipulatives can oer dierent learning modalities
to accommodate dierent learning styles [60, pp. 35-41][61]. Paek demonstrates “how vir-
tual manipulatives, designed to provide multimodal interactions, support richer perceptual
experiences that promote conceptual learning” [62, p. nulla]. Multimodal interaction refers
to the multiple ways (modes) with which users interface with learning content.
In formal education, instructional manipulatives may be more or less ecient based on
each student’s learning style. Sarama and Clements caution us that “students may hold,
move, and arrange physical objects without thinking about the concepts” [63, p. 74]. How-
ever, as Narayanan et al. point out, “the success of any educational tool or technique largely
depends on its successful adoption by both the teachers and students who are the primary
stakeholders” [64, p. 299]. Students conceptualize aordances for manipulatives based on
available semiotic modes [65, pp. 39-41]. Although physical instructional manipulatives may
oer dierent semiotic resources than virtual manipulatives, a virtual simulation of a phys-
ical manipulative could allow the physical and the virtual manipulatives to share learning
resources.
1.2 Aective States
1.2.1 Hedonic Tone & Pleasure-Arousal Theory
Hedonic Tone is a measure of the ability to feel pleasure, while anhedonia is the inability
to feel pleasure. Keller et al. “found that trait anhedonia was negatively correlated with
pleasantness ratings of music stimuli” [66, p. 1319]. Experiencing artwork and making art
are known activities with a potential for enjoyment (hedonic usability). Making art is a
self-expressive activity. This dissertation proposes that instructional activities based on
self-expressive outputs, such as musical expression, could be more engaging due to their
enjoyment potential.
4
The theory of pleasure-arousal conceptualizes a two-dimensional aective state [67]. The
two dimensions are the intensity of pleasure, and the intensity of arousal. Pleasure indicates
enjoyment, while arousal indicates attentiveness to an experience (incoming stimuli). In
recent studies, electro-dermal activity shows evidence that individuals can have multiple
states of arousal. For example there have been observed dierences between their left and
right hands [68]. This dissertation focuses on the conscious and subjective perspective of
people experiencing an instructional activity. Therefore, the dissertation’s study relies on
self-reported measures of aective states.
Betella and Verschure present a measure of pleasure and arousal for ”digital self-assessment,”
which is useful for instructional systems [69]. The measure consists of two sliders for the
self-assessment of pleasure and arousal. In the dissertation’s study, the concept of pleasure
is narrowed down to the feeling of enjoyment, and the concept of arousal is narrowed down
to the feeling of excitement during an instructional activity.
1.2.2 Motivational Salience
Motivational salience is a measure of an individual’s feeling of encouragement (positive) or
discouragement (negative) to continue an activity. Zimmerman et al. believe that “perceived
ecacy to achieve motivates academic attainment both directly and indirectly by inuencing
personal goal setting” [47, p. 674]. In other words, learners are self-motivated based on their
perception of being able to achieve the challenges they face. Informal education may leverage
the interests of learners, but formal education is not as exible. Wlodkowski argues “that
with student motivation, when only one thing goes wrong, the entire process may come to
a complete stop” [70, p. 12]. Monitoring learners’ aect during instructional activities could
be used to assess the eciency of the activities [71].
Formal education operates within a schooling framework that does not make it easy
to customize educational paths. In the context of primary and secondary education, Druin
discusses the child student “as user,” “as tester,” “as informant,” and “as design partner” [72,
5
pp. 3-13], and quotes a child that expresses the desire to have fun while learning by playing
with friendly and interesting tools. To assess academic motivation, Jones developed the
“MUSIC Model of Motivation,” a measure “that can be used by instructors to design courses
that will engage students in learning.” This model suggests “a social-cognitive theoretical
framework, in which ve factors lead to increased student motivation, resulting in increased
student learning.” The ve factors are: “eMpowerment,” “Usefulness,” “Success,” “Interest,”
and “Caring” [73, p. 273].
1.3 Self-Expression
1.3.1 Communication
Conscious learning includes communication as a way of making meaning. The principle of
embodiment unies aect and cognition. It assumes that conscious perception is regulated by
the nerve sensorimotor system [74,75]. Conscious perception is therefore subject to the state
of the sensorimotor system. This means that unconscious learning encapsulates conscious
learning. When communicating via languages human sensorimotor processes the signier
(stimulus) unconsciously, and cognition processes the signied (meaning) consciously.
In semiotics theory, visual communication can involve an iconic sign system [76], and also
a symbolic sign system through written language. Both symbolic and iconic sign systems
are semiotic resources; however icons are referents themselves, while symbols have external
referents expected to be present in people’s knowledge. For example, with spoken or written
language, a person could communicate the same message (words) to two dierent people
who may make sense of two dierent messages depending on their individual knowledge and
their physical state.
Another sign system, the indexical (deictic), is eectively incomplete. Specic context
is necessary to specify the signied less abstractly, and thus to make more specic sense of
such communication [77]. For example, the third study refers to a study that in some specic
6
context is third. This dissertation refers to itself, while a dissertation refers abstractly to
the concept of a dissertation. Dierent modes of communication could be constructed by
blending dierent sign systems [78].
1.3.2 Musical Expression
While listening to and making music are both a form of self-expression, making music depends
on planned behavior (goals) for the creation of a musical structure. Here, it is important to
dierentiate between musical expression and musical expressiveness. The former pertains to
the production of musical signals, while the latter pertains to the capability of dierentiating
a musical signal.
Personal computers have made it easy for people to record and process sound using o-
the-shelf equipment. The paradigm of the music sequencer (software) enables individuals
to compose and produce music without physical musical instruments. However, another
paradigm, the real-time synthesis engine, enables individuals to redesign the sound produc-
tion while the sound is being generated.
This dissertation develops an instructional apparatus based on the real-time synthesis
engine paradigm to enable users to practice computational thinking through sound produc-
tion.
1.4 Research Characterization
The learning modalities of instructional systems for the practice of computational thinking
are under-investigated [23,78,56]. This dissertation designed and performed an exploratory
study of learning modalities for the practice of computational thinking.
The dissertation’s study combined exploratory, descriptive, and correlation research into
an experiment for the exploration of learning modalities among general public users. The
study gathered demographic data and descriptive statistics in order to analyze the relations
7
between measures of aective states and computational knowledge within for groups of par-
ticipants. Because of its exploratory character and limited sample, the study’s objective was
not to create normative data. Rather, it was to use empirical data to inform instructional
systems design theory.
1.5 Methodological Contributions
This dissertation presents a methodological approach in instructional systems design for the
practice of computational thinking through computing for self-expression. The proposed
methodology blends visual programming with sound production and with physical-virtual
manipulatives to develop a semiotic multimode for the practice of computational thinking.
The importance of this research is in its potential to provide new and improved strate-
gies for introducing computational thinking practice in formal education via practical and
potentially enjoyable instructional activities. The proposed methodology includes a novel
method for the assessment of computational thinking.
More specically, the methodological contributions of this dissertation are:
1. A novel nomological network for the construct of Computational Thinking. The nomo-
logical network is premised on the “bounded rationality” principle [79, pp. 697-699];
2. An experimental investigation of the aective quality of visual and musical modalities
in the context of computational cognitive practice;
3. A novel method of instructional assessment based on predened levels of thinking and
knowledge. The proposed method is exemplied using Bloom’s revised taxonomy [80],
and
4. A novel methodology for the development of physical-virtual manipulatives for the
practice of computational thinking that focuses on sharing learning resources between
a physical manipulative and its virtual simulation.
8
2 Literature Review
2.1 Embodied Cognition
2.1.1 System-View of Embodiment
Action within an environment leads to a sense-making process that has the potential to
develop new, and to reshape prior knowledge [59]. Meier et al. comment that “current
embodiment research in social psychology typically aims to identify whether a concept or
related metaphor is embodied” [81, p. 5]. Communication among people is based on semiotic
resources. When an idea is expressed through semiotic resources, then it is theorized that
the physical object(s) mediating these semiotic resources embody that idea.
Johnson and Rohrer describe embodied cognition by summarizing how some “pragmatist
philosophers viewed cognition” [82, p. 2]:
(1) Embodied cognition is the result of the evolutionary processes of variation, change, and selection.
(2) Embodied cognition is situated within a dynamic ongoing organism-environment relationship.
(3) Embodied cognition is problem-centered, and it operates relative to the needs, interests, and values
of organisms.
(4) Embodied cognition is not concerned with nding some allegedly perfect solution to a problem, but
one that works well enough relative to the current situation.
(5) Embodied cognition is often social and carried out cooperatively by more than one individual organ-
ism.
In the context of the pragmatism view, instructional systems should be capable to eval-
uate how well they work for their users [83]. Theoretically, instructional systems that oer
multiple semiotic modes, could create multiple learning pathways for users, even though
not all of them may prove helpful. Self-regulated learning depends on having the ability to
choose learning paths that make more sense and are more desirable. Design principles for
instructional systems depend on the system’s conceptual components.
Most of what the human body perceives with its senses does not pass into conscious
awareness. Rather, stimuli are aggregated before being processed on the conscious level.
Varela et al. make two points about the inuence of bodily actions on cognition: “rst,
that cognition depends upon the kinds of experience that come from having a body with
9
various sensorimotor capacities, and second, that these individual sensorimotor capacities
are themselves embedded in a more encompassing biological, psychological, and cultural
context” [84, p. 173]. Gibbs identies three system-views of embodiment:
1. “the neural level,”
2. “the phenomenological conscious experience,” and
3. “the cognitive unconscious” [85, pp. 39-40].
Brain imaging techniques create snapshots of brain activation to describe neurological
patterns. In the neural perspective, cognition is embodied. This means that the accuracy
and precision of awareness is inuenced by sensorimotor activity. Aleksander et al. discuss
the human “mind’s eye” as well as the “learning and remembering” to point out that “the
sensory strip in the cortex” and “patterns in the brain are related to where bits of our body
are and what they are sensing” [86, pp. 162-165]. Debarba et al. point out that “exper-
imental protocols have shown that the sense of embodiment is much more malleable than
commonly assumed” [87, p. 2]. A summary of the system-view of neural-level embodiment
is oered by Gaiseanu, who links it to consciousness. Which “is mainly manifested by data
accumulation, informational operability, emotional reactivity, functional self-control, asso-
ciativity and creativity, self-condence, on the basis of the genetic inheritance of species,
received from the parents” [88, p. 14]. Literature of neuroscience oer in-depth explana-
tions of the embodiment principle. For example, Johnson and Rohrer argue that “one of
the most profound ndings in neuroscience is that nervous systems exploit topological and
topographic organization” [82, p. 7].
The phenomenological conscious experience discusses the sense-making process, which
“includes all our unconscious knowledge and thought processes” [85, p. 40]. Gallagher and
Zahavi argue that “the rst-person point of view on the world is never a view from nowhere;
it is always dened by the situation of the perceiver’s body, which concerns not simply
10
location and posture, but action in pragmatic contexts and interaction with other people”
[89, ch. 4].
The cognitive unconscious refers to the information processing occurring in the human
body without conscious awareness of it. Gibbs sums a system-view of cognitive unconscious
embodiment as “all the mental operations that structure and make possible conscious ex-
perience, including the understanding and use of language” [85, p. 40]. Kihlstrom points
out that “research on perceptual-cognitive and motoric skills indicates that they are autom-
atized through experience, and thus rendered unconscious.” “In conversational speech, for
example, the listener is aware of the meanings of the words uttered by the speaker but not
of the phonological and linguistic principles by which the meaning of the speaker’s utterance
is decoded” (phonological as in speech sounds) [90, p. 1447]. The cognitive unconscious
system-view is concerned with identifying the unconscious processes that enable conscious
awareness.
2.1.2 Notions of Embodiment
This section briey presents “dierent notions of embodiment” that are discussed by Ziemke
in more detail [74, p. 1306]. In the context of instructional systems design, each notion
promotes dierent design principles for instructional systems.
Structural Coupling
The structural coupling refers to the relationship between the environment and any indi-
vidual agent it encapsulates. In such interactive ecosystem, both can aect each other’s state.
Structural coupling may occur via physical or virtual means. Virtual reality could enable
the communication of an agent with computer generated and remote physical environments.
In this dissertation, the coupling between an agent and its surrounding environment is la-
beled natural, while the coupling, via virtual means is labeled articial. Further, predened
agentive or environmental behaviors are labeled xed, while dynamic behaviors are labeled
dynamic.
11
Historical Evolution
With time, a structural coupling is undergoing sequential changes. In other words, struc-
tural couplings may have a history of changes that characterize the conditions during these
changes. More specically, a progression of changes reveals which endpoint of the coupling
initiated an adaptive change in response to the other endpoint’s behavior. The embodiment
principle postulates that the cognitive development of an agent is dicult to distinguish from
(confounded with) the historical evolution of the environment encapsulating that agent.
Social Interaction
A social interaction is based on the notion of recurring environmental conditions. This
notion of embodiment views social links to surrounding agents as part of the environment a
focal agent is coupled with. The focal agent perceives other agents’ behavior as environmen-
tal conditions [91,92]. Thus, each agent although encapsulated by the same environment
expresses a distinct interpretation of the same environmental conditions.
Physical Artifact
A physical artifact reacts to its surrounding environment. The key aspect of this notion
of embodiment is the encapsulation of a physical structure by its surrounding environment.
Passive artifacts react to environmental change, while active artifacts enact towards prede-
termined goals with or without environmental change.
Automaton
An automaton presumes self-actuating capability. This concept is similar to an active
physical artifact. The dierence between the two is that an automaton is capable of adap-
tive behavior based on some dynamic model of sensorimotor abilities. Conceptually, the
automaton is the blend of an active physical artifact and historical evolution. Nevertheless,
an automaton does not learn how to interpret the same stimulus in dierent ways.
Organism
An organism, however, learns how to interpret the same stimulus dierently based on ex-
12
perience (historical evolution). Its behavior is driven by sense-making of both internal and
external stimuli. This means that an organism is capable of altering its decision making rule-
set (logic). Johnson and Rohrer point out the social aspect of the “organism-environment
coupling” and argue that for organismic cases “starting with single-cellular organisms and
moving up by degrees to more complex animals. In every case we can observe the same
adaptive process of interactive co-ordination between a specic organism and recurring char-
acteristics of its environment” [82, p. 6].
2.1.3 Embodiment in Learning Theories
Behaviorism theory conceptualizes learning as a measurable process. When learners suc-
cessfully achieve predened objectives, then they exhibit the abilities necessary for achieving
these objectives. Therefore, the successful performance of predened objectives serves as a
measure of learning. Behaviorism seems in alignment with the neural-level system-view of
embodiment [93].
Cognitivism theory conceptualizes learning as information meta-processing. Learners
make sense of perceived information, and then create or reshape knowledge to be memo-
rized. Cognitivism’s perspective disagrees with behaviorism’s view of thinking as a behavior,
because thinking is conceptualized as a driver of behavior. Cognitivism seems in alignment
with the phenomenological conscious experience system-view of embodiment [94].
Connectivism theory conceptualizes learning as information gathering from a network of
physical and digital sources. It focuses on the means and modes of retrieving information,
rather than on cognitive mechanisms. A principle of connectivism is that there are multiple
styles of learning. Moreover, informal learning is emphasized as the norm in the digital era
of the internet. This perspective seems in alignment with the phenomenological conscious
experience system-view of embodiment [95, pp. 53-68].
Constructivism theory conceptualizes learning as the sense-making of personal experience.
People construct meaningful ideas based on how what they are experiencing makes sense to
13
them. In such perspective, predened learning objectives should describe both the context
and content of learning. Constructivism seems in alignment with the unconscious cognition
system-view of embodiment [96].
Humanism theory conceptualizes learning as a self-dened process according to individual
ability and values. Learning should be customized for the ability and objectives of the
learner. Such approach suggests that education should facilitate self-regulated learning based
on learners’ predened objectives. Humanism seems in alignment with the unconscious
cognition system-view of embodiment [97].
In the context of situated learning, the interaction of learners with the learning content is
moderated. Hwang and others describe and discuss the “situated reective learning model”
[98, p. 142]. Reection is a learning principle in the theory of constructivism. The theory of
connectivism suggests that a single semiotic resource provides a single communication chan-
nel, while more than one semiotic resource could provide multiple communication channels.
Instructional systems that oer multiple communication channels may appeal to a broader
community of learners.
There are some fundamental questions when designing instructional systems:
a. Will it be possible to assess users on what are desired semiotic modes before presenting
learning content to them?
b. Will the system be able to learn from users’ reections?
Weliweriya and others argue that “to better represent an idea or a concept, students should
be able to strategically combine multiple semiotic resources” [99, p. 3]. This dissertation
presumes that more users will be able to explore and express ideas when practicing with
systems that enable them to choose from a variety of available semiotic resources [100,
pp. 233-249].
The technological progress made small-scale robotics cheaper and commercially accessi-
ble. Kennedy et al. argues that “the application of social robots to the domain of education
14
is becoming more prevalent” [101, p. 293]. Simple robotics could be used as learning manip-
ulatives in education that oer programmable features based on their degrees of freedom.
Fischer et al. researched the interaction between humans and robots and their “investigation
has shown that not only the robot’s physical embodiment, but also its degrees of freedom
inuence human-robot interaction” [102, p. 469]. Instructional activities for cognitive prac-
tice could exploit robotic systems as instructional platforms.
There are many theories of cognition. Newell suggests what should a unied theory of
cognition address [103, p. 15]:
Problem solving, decision making, routine action
Memory, learning, skill
Perception, motor behavior
Language
Motivation, emotion
Imagining, dreaming, daydreaming, …
Newell also describes a learning theory, “the SOAR Qualitative Theory of Learning” [103,
p. 317]:
1. All learning arises from goal-oriented activity
2. Learn at a constant short-term average rate .5 chunk/sec (the impasse rate)
3. Transfer is by identical elements and will usually be highly specic
4. Learn the gist of a sentence, not its surface structure, unless focus on it
5. Rehearsal helps to learn, but only if do something (depth of processing)
6. Functional xity and Einstellung will occur
7. The encoding specicity principle applies
8. The classical results about chunking (such as for chess) will hold
“Einstellung” refers to a set of problem-solving techniques that are prior knowledge for
an individual. The Einstellung eect is the predisposition of each individual to attempt
problem-solving based on their prior knowledge in problem-solving. The “chunking” refers to
learning in terms of short-term realizations of facts, concepts, procedures, and self-reections,
while “transfer” refers to the application of prior knowledge and skills to new contexts [103,
p. 317].
15
2.2 Sense-Making
2.2.1 Semiotic Resources
Semiotic resources are the building-blocks of semiotic modes. However, not all semiotic
resources have the same level of preciseness when it comes to communicating meaning.
Dierent types of semiotic resources use dierent types of communicational signage. Three
basic types of signage are labeled to describe the relation between signier and signied:
iconic, indexical, and symbolic. An icon-sign is supposed to resemble the form of the signied.
An index-sign should resemble an indication of the signied such as an antecedent or a
consequent. Lastly, a symbolic-sign is linked to meaning by social convention. Individuals
must learn symbolic relationships in order to interpret symbolic-signs.
2.2.2 Semiotic Modes
Asemiotic mode is a cultural structure of semiotic resources. Consequently, a semiotic
multimode is a coherent aggregate of semiotic modes. Nigay and Coutaz dene a semiotic
mode with a software engineering perspective: “mode refers to a state that determines the
way information is interpreted to extract or convey meaning” [9, p. 172]. A multimodal
system operates based on one or more multimodes. The approach to multimodal analysis
diers depending on the objective of the analysis.
Jewitt discusses “three approaches within multimodality” that “can be roughly cate-
gorized” as “a social semiotic approach to multimodal analysis,” “a systemic functional
grammar (SFG) multimodal approach to discourse analysis,” and “multimodal interactional
analysis” [104]. This dissertation borrows elements from the rst and the last approaches.
Social semiotics is useful in analyzing a visual learning modality, while the interaction anal-
ysis is useful in analyzing an auditory learning modality.
Multimodality is often used to describe the input modes for human-computer interaction,
such as touch, speech, and motion gestures. However, the concept of multimodality is
16
not limited to input-data analysis. In the context of instructional activities, multimodality
characterizes the system’s semiotic resources that users are exposed to. Users engaging with
a multimode do not necessarily focus on each individual mode. Nevertheless, each individual
mode provides stimuli for learners to focus their attention to.
2.2.3 Inter-Relations of Semiotic Resources
An inter-semiotic resource relationship is one or more relations between two or more semi-
otic resources. Inter-semiotic relationships among modes emerge from relations between the
attributes of semiotic resources. Instructional design could leverage inter-semiotic relation-
ships to combine modes into multi-modes. For example, playing a violin in an apartment
can be loud for neighbors. Playing an electric violin can be even louder when using loud-
speakers, but it can also be much quieter when using headphones. However, the goal of
sound production is the same when playing the electric violin using either loudspeakers or
headphones.
Inter-semiotic relationships instigate “syllogisms” across semiotic modes [103, pp. 378-
410]. For example, empirical observation of discrepancies between expected and observed
system behavior can instigate reections on false expectations. Heuristic learning takes place
when instructional system users troubleshoot unexpected system behavior.
2.2.4 Aordances of Semiotic Modes
Each aordance of a semiotic mode is a possible and straightforward way of applying the
semiotic mode in a task. Francesconi argues that “some of the pillars of the embodiment
paradigm that correspond—in spite of the classical cognitivism—to key concepts in edu-
cation are the role of subjective experience (rst-person perspective), the body (embodied
cognition), the environment (embedded cognition), and the situation/ experience (situated
cognition)” [105, p. 264]. No matter which aordances were intended by semiotic mode de-
signers, learners may not recognize them.
17
An assumption of the embodiment principle is that individuals who pay more attention to
certain semiotic resources will be biased towards these resources during their sense-making.
Similarly, individuals are biased towards the aordances they perceive, thus ignoring aor-
dances they fail to perceive. For example, an individual pays more attention to the lyrics of
a song, while another individual is more attentive to instrumental musical elements of the
same song. In another example of complementary resources, individuals procient in the
spoken language may pay little attention to captions of audiovisual content.
Deixis is the use of deictic expressions when describing events or ideas. For example,
consider the question: do you like this? What does the word this denote when it is used
as a demonstrative pronoun? The meaning depends on what is being demonstrated by the
person asking the question. An assumption of deictic cognitive function is that sense-making
is coupled to sensorimotor [106,107,108]. Human languages use deictic words to reference
objects by means of demonstration. Bergen and Plauche label and exemplify a few linguistic
constructions and their relational features [109, p. 34]:
i. Innitival existential There’s the shopping to do.
ii. Ontological existential There is a Santa Claus.
iii. Presentational existential There walked into the room a camel.
iv. Evaluative existential There’s brie and then there’s brie.
v. Strange existential There’s a man been shot.
2.3 Computational Thinking
2.3.1 Conceptual Denitions
It is important to identify denitions that are helpful in introducing computational thinking
both in formal and informal educational settings. Kale et. al dene computational thinking
with the following steps: “Confrontation,” “Decomposition,” “Pattern recognition”; “Ab-
straction”; “Algorithm/Automation,” and “Analysis” [110]. Borges et al. describe the “rela-
tions between formal thinking and computational thinking” (the formal operational thinking
stage of cognitive development ranges from around 11 years to adulthood) and describe the
“computational thinking elements related to digital fabrication activities” as: “Algorithm
18
thinking,” “Abstraction,” “Decomposition,” “Generalization,” “Evaluation,” and “Data ma-
nipulation” [111]. Contrary to skills-based denitions of computational thinking, Wing de-
nes computational thinking as “the thought processes involved in formulating a problem
and expressing its solution(s) in such a way that a computer —human or machine— can
eectively carry out.” Wing also points out that “the most important and high-level thought
process in computational thinking is the abstraction process” [112, p. 8].
In appendix 6, table 15 presents denitions of, and perspectives on computational thinking
from relevant literature. These ndings are grouped under dierent labels to describe a
group’s common characteristic. Rose et. al. report on common concepts in the denitions
of computational thinking found in their literature review [113, p. 299]:
• Abstraction and generalisation (removing the detail from a problem and formulating solutions in
generic terms)
• Algorithms and procedures (using sequences of steps and rules to solve a problem)
• Data collection, analysis and representation (using and analysing data to help solve a problem)
• Decomposition (breaking a problem down into parts)
• Parallelism (having more than one thing happening at once)
• Debugging, testing and analysis (identifying, removing and xing errors)
• Control structures (using conditional statements and loops)
Instruction is sucient to introduce concepts, but insucient to develop skill. While
concepts need to be understood, skills need to be practiced. Harel and Koichu characterize
“learning as a multi-dimensional and multi-phase change occurring when individuals attempt
to resolve what they view as a problematic situation” [114, p. 115]. Designing instructional
activities should not simply be a matter of delivering information, but also an environment
to with which to apply the instructional content. Authors Barr and Stephenson, Roman-
Gonzalez et al., Yadav et al., Curzon et al., Harel and Koichu, Gibson, and Bransford
et al. have developed computational activities for the practice of computational thinking
[46,115,116,117,114,118,119].
In the context of the visual-dataow programming paradigm, Turchi and Malizia point
out that “visual-programming environments […] are relatively easy to use and allow novices
19
to focus on designing and creating while avoiding the issues of the traditional murky and
complicated programming syntax.” The authors introduced “TAngible Programmable Aug-
mented Surface (TAPAS), a system that allows users to adapt a public display’s features
to their own needs, by using the movements of their smartphone to interact with it” [120,
pp. 2-3]. Visual-programming is based on the paradigm of digital abstractions [121,122].
Visual-dataow programming renders data connections visible by mapping them out on
screen.
Practicing computational thinking within a creative context, such as music making, may
help learners stay motivated to practice due to the self-expressive creative output. Hidi and
Renninger argue “the level of a person’s interest has repeatedly been found to be a powerful
inuence on learning” [123, p. 111].
2.3.2 Quantication
Dierent ways for the quantication of computational thinking have been suggested and
tested, and few of them even implement automated assessment. Moreno-Leòn and Robles
developed “a web tool to automatically evaluate scratch projects” [124]. An approach to
real-time automated assessment of skills related to computational thinking is discussed by
Koh et al. and is based on recognizing design patterns [125,126,127].
Another approach by Moreno-Leòn et al. developed software to calculate “Halstead’s met-
rics” for the evaluation of Scratch programs [128, p. 1040]. Halstead’s metrics were developed
to measure software complexity, thus the authors used software complexity as evidence of
knowledge on computational practices [129]. Such approach implicates the measurement
of computational thinking by combining it with the ability to express it through a specic
programming environment. The measurement of computational thinking through specic
programming environments is a common practice. Atmatzidou and Demetriadis report that
“the dierent modality (written and oral) of the Computational Thinking skill assessment
instrument may have an impact on students’ performance” [18, p. 661]. Curzon et al. be-
20
lieve that assessing skills related to computational thinking “can be done using an adapted
version of the existing subject framework for the computing subject itself” [117, p. 7]. This
means that each instructional activity should also provide its own measure of computational
thinking specic to that activity.
Korkmaz et al. developed a measure “that could be used in the identication of the
computational thinking levels of students” [130, p. 568]. This measure collects self-reported
data, and the authors perceive computational thinking as “a method of problem solving,
system designing and also a method of understanding the human behaviors by drawing
attention to the basic concepts of the science of computer” [130, p. 558]. A general approach
to measuring computational thinking can be achieved using predened levels of knowledge
and thinking.
A problem with measuring self-reported data is that it weakens the measurement’s objec-
tivity. Additionally, self-reported data may be misleading due to social and personal factors.
Also, creative programming outputs do not tend to be as optimized as software engineering
outputs. Wangenheim et al. believe that “in practice it may be dicult to provide per-
sonalized, objective and consistent feedback.” Wangenheim et al. based their computational
thinking measure on “executing the code and comparing the generated output to the control
output” [131, p. 125].
Ioannidou et al. report that with a “phenomenalistic analysis for thousands of games and
simulations developed by AgentSheets users over the years, we created a list of basic and
advanced Computational Thinking Patterns” [132, p. 6]. These patterns enabled a semantic
characterization of code submitted to the AgentSheets platform. Chang et al. used language
recognition to assess computational thinking based on weights of programming blocks in the
Scratch visual-programming environment [133]. To promote assessment automation, Fuentes
Pérez et al. used “amCharts” to present data by graphics that “can be downloaded, exported,
or printed from the platform itself” [134, p. 792].
21
2.3.3 Educational Applications
Kalelioglou et al. reviewed 125 papers of relevant literature and suggest that “it can be
benecial to teach Computational Thinking by starting discussions on the following: how to
teach Computational Thinking skills, how to assess if our students really have Computational
Thinking skills, and how to assess if our students can adopt Computational Thinking skills
into real-life situations” [135, p. 592]. These suggestions imply that computational thinking
depends on levels of knowledge and skill. While knowledge can be communicated, skills must
be practiced. Hence, computational thinking must be practiced.
In the context of formal education, Silva et al. argue “teaching computational thinking
is directly associated with constructionism and is essential for the complete – and not only
supercial – assimilation of knowledge. Computational thinking is related to the ability
to abstract knowledge in dierent dimensions, while constructionism emphasizes the impor-
tance of shaping absorbed knowledge” [17, p. 285]. The authors also report that “approaches
that aim to introduce computational thinking to high school students using unplugged com-
putation are unknown” [17, p. 288].
What skills should be practiced to develop computational thinking? Lockwood and
Mooney review relevant literature and report seven-stages from elementary to advanced
skills related to computational thinking [136, p. 26]:
1. “Animations – watching a movie or similar”
2. “Interactive simulations – a simulation which the user can alter certain parameters”
3. “Collective simulations – like above but with a social element”
4. “Construction set simulations – construction kits used to solve domain-specic problems”
5. “Pattern based authoring – begin to design the behaviours of simulations actors”
6. “End user programming – using tools like AgentCubes or Scratch”
7. “Traditional programming – using languages such as Java or C++”
Moreno-León and et al. specify a weight for each skill related to computational thinking.
The authors determine that the eects of computational thinking are explained “27%” by
“reasoning ability, visual ability, verbal ability, and numerical ability,” while “24%.” are
explained by “non cognitive personality factors.” The remaining “49%” remains unexplained,
22
and the authors conclude that “the best scientic knowledge on the topic to this date seems
to indicate that the most eective way to foster computational thinking from early ages is
by including programming activities in the school curriculum” [137, p. 1685].
How could skills be practiced? Hsu et al. report “16 learning strategies” [138, p. 302]:
Project-based learning, problem-based learning, teacher-centered lectures, collaborative learning, game-
based learning, aesthetic experience, concept-based learning, systematic computational strategies, scaf-
folding, problem-solving systems, storytelling, embodied learning, universal design for learning, HCI
teaching, design-based learning, and critical computational literacy.
How could formal educational activities be assessed? Hsu et al. argue “because dierent
learning strategies and subjects will be applied at dierent ages, formal and informal courses
also need dierent scoring guidelines. Such assessment may be of help later in designing
teaching activities and modifying teaching strategies” [138, p. 308]. Ilic et al. “determined
that parametric analysis techniques and content analysis techniques were the most frequently
used data analysis methods in the reviewed studies on Computational Thinking” [139, p. 146].
The authors also suggest that Computational Thinking research has focused on education
and instructional technology and their “main ndings in studies conducted on Computational
Thinking” include “ndings on the position of Computational Thinking in curricula and
education” [139, p. 142]:
1. Computational Thinking should be integrated to education
2. Curricula should be reorganized based on Computational Thinking skills
3. Computational Thinking is useful in courses
4. Computational Thinking should be included in teacher training
Formal education lags behind informal education in utilizing instructional activities based
on robotic platforms. Ioannou and Makridou discuss “educational robotics” [140, p. 2533]:
In terms of assessment, eorts have been made to assess Computational Thinking in the context of using
visual languages to teach programming and Computational Thinking skills. Among others, Koh et al.
(2010) proposed another two approaches: The Program Behavior Similarity (PBS) and the Computa-
tional Thinking Pattern Graph (CTPG) in which student-created games and simulations are analyzed
towards depicting the Computational Thinking concepts implemented by the students.
Lastly, the blend of visual and text-based programming can be very powerful for learn-
ing, because once an abstraction has been designed in a text-based environment, then it can
23
be represented visually by means of inputs, attributes, and outputs. Massive Open Online
Courses (MOOCs) have made it easy to seek knowledge online. Free/Open Source Software
(FOSS) oers free and easy access to practice through digital media. As Hanna points out,
“reading masterful source code teaches the student good programming habits” [37, p. 5].
Also, Lu and Fletcher suggest “the emphasis should be on understanding (and being able to
manually perform) computational processes, and not on their manifestations in particular
programming languages” [141, p. 261]. Most visual-programming environments do not help
users to transition to text-based programming environments. Vanvorce and Jamil discuss
the transition from visual to syntax-based programming. In their approach, “codeMapper,”
“students write codes or import codes in text-based languages, and thus they must be some-
what familiar with the syntax” [142, pp. 2-3].
In Appendix 6, Table 16 reports on the teaching modules used in curricula of compu-
tational thinking pertaining to all education levels, and even to professional development.
Additionally, Table 17 describes educational tools and their respective components or activ-
ities aiming to the development of skills commonly related to computational thinking.
2.3.4 Gamication and Musical Expression
Playing games is an enjoyable activity for many, particularly for children and teens. For
this reason, Boulton et al. propose that “game jams can support engagement with informal
learning beyond schools across a range of disciplines, resulting in an exciting experience
associated with strong, positive emotions which can signicantly support learning goals”
[143]. In another example of practicing computational thinking through a game, Berland
and Lee suggested “code categories” that require computational thinking during a game
of “Pandemic”: “Conditional logic,” “Algorithm building,” “Debugging,” “Simulation,” and
“Distributed computation” [144, p. 70].
Serious games inform players how well they are progressing towards predened learning
objectives. A game genre aiming to the development of skills transferable to the physical
24
world is simulation training. However, less realistic serious games could involve transferable
cognitive skills, such as conditional logic in decision-making. Due to their easy distribu-
tion and often low cost, digital games dominate the serious-gaming eld. Kazimoglu et
al. discuss what tasks were mapped to digital game activities for the development of skills
related to computational thinking: “Problem identication and decomposition,” “Creating
ecient and repeatable patterns,” “Practicing debug-mode,” “Practicing run-time mode,”
and “Brainstorming” [21, p. 528].
Weintrop et al. “dene constructionist video games as: Designed computational envi-
ronments in which players construct personally meaningful artifacts to overcome articial
conict or obstacles resulting in quantiable outcomes” [145, p. 4]. The authors build games
using the following three principles [145, pp. 5-6]:
Principle 1: Constructionist video games include suciently expressive construction tools for players to
engage with and build personally meaningful artifacts.
Principle 2: Game goals and construction tools encourage exploration and discovery during play.
Principle 3: Learners engage with and employ powerful ideas to advance through the game.
Commonly used visual programming environments for educational game development are:
1. Alice (https://www.alice.org/)
2. Blockly (https://developers.google.com/blockly/)
3. mBlock (http://www.mblock.cc/)
4. Scratch (https://scratch.mit.edu/)
5. ScratchJr (https://www.scratchjr.org/)
Bell and Bell discuss “ways to integrate computational thinking and music, and to show
how arts can have a primary role in Computational Thinking learning” [146, p. 165]. The
authors note that “the elements of music that dene the scope of curriculum are often
articulated as a list such as pitch, timbre, texture, dynamics, duration, tempo, and structure”
[146, p. 153].
25
Programming musical expression depends on available programmable expressiveness, which
includes coded structures that parameterize expression in some way. More broadly, music
theory is knowledge on computational practices in sound organization, and music notation
is a semiotic resource. Widmer and Goebl discuss computational models for musical expres-
siveness. The authors argue that “the purpose of computational models of expressive music
performance is thus to specify precisely the physical parameters dening a performance (e.g.,
onset timing, inter-onset intervals, loudness levels, note durations, etc.)” [147, p. 204].
Music composition is essentially a process of computational thinking that aims to structure
sound production around extemporized and conventional musical elements. For example,
the development and use of antecedent and consequent musical sequences. Music theory is
knowledge that comes from analyzing what was conventionalized through music composition
in the past. New technology, such as virtual musical instruments and other ways of processing
sound can lead to new conceptual ideas and thus the development of new music theory and
novel ways of musical expression.
In the context of algorithmic music, Edwards argues “using algorithmic-composition tech-
niques does not by necessity imply less compositional work or a shortcut to musical results;
rather, it is a change of focus from note-to-note composition to a top-down formalization of
compositional process […] perhaps counterintuitively, such formalization of personal compo-
sition technique allows the composer to proceed from concrete musical or abstract formal
ideas into realms hitherto unimagined, sometimes impossible to achieve through any other
means than computer software” [148, p. 67]. Software that enables sound processing has
been instrumental in creating new ways to compose music and use machines for musical
expression. Commonly used visual programming environments for sound production are:
1. Audulus (http://audulus.com/)
2. Max (https://cycling74.com/products/max/)
26
3. OpenMusic (http://repmus.ircam.fr/openmusic/home)
4.