Content uploaded by Marcos Román-González
Author content
All content in this area was uploaded by Marcos Román-González on Jan 19, 2017
Content may be subject to copyright.
Does computational thinking correlate with personality?
The non-cognitive side of computational thinking
Marcos Román-González
Universidad Nacional de Educación a
Distancia (UNED)
Facultad de Educación, C/ Juan del
Rosal, nº 14, 28040, Madrid (Spain)
Tel.: +34 91 398 90 37
mroman@edu.uned.es
Juan-Carlos Pérez-González
Universidad Nacional de Educación a
Distancia (UNED)
Facultad de Educación, C/ Juan del
Rosal, nº 14, 28040, Madrid (Spain)
Tel.: +34 91 398 69 55
jcperez@edu.uned.es
Gregorio Robles
Universidad Rey Juan Carlos (URJC)
ETSI Telecomunicac., Departamental
III, Camino del Molino s/n, 28943
Fuenlabrada (Madrid, Spain)
Tel.: +34 91 488 87 50
grex@gsyc.urjc.es
Jesús Moreno-León
Programamos.es & URJC
ETSI Telecomunicac., Departamental
III, Camino del Molino s/n, 28943
Fuenlabrada (Madrid, Spain)
Tel.: +34 91 488 87 50
jesus.moreno@programamos.es
ABSTRACT
Computational thinking (CT) is being considered as a key set of
problem-solving skills to be acquired by the new generations of
digital citizens and workers in order to thrive in a computer-based
world. However, from a psychometric point of view, CT is still a
poorly defined psychological construct: there is no full consensus
on a formal definition of CT or how to measure it; and its
correlations with other psychological constructs, whether
cognitive or non-cognitive, have not been completely established.
In response to the latter, this paper aims to study specifically the
correlations between CT and the several dimensions from the ‘Big
Five’ model of human personality: Conscientiousness, Openness
to Experience, Extraversion, Agreeableness, and Neuroticism. To
do so, the Computational Thinking Test (CTt) and the Big Five
Questionnaire-Children version (BFQ-C) are administered on a
sample (n = 99) of Spanish students from 5th to 10th grade.
Results show statistically significant correlations between CT and:
Openness to Experience (r = 0.41), Extraversion (r = 0.30), and
Conscientiousness (r = 0.27). These results are partially consistent
with the literature about the links between cognitive and
personality variables, and corroborate the existence of a non-
cognitive side of CT. Hence, educational interventions aimed at
fostering CT should take into account these non-cognitive issues
in order to be comprehensive and successful.
CCS Concepts
• Social and professional topics➝ Computational thinking
• Social and professional topics➝ K-12 education
Keywords
Computational thinking; Personality; Computational Thinking
Test; Big Five model; Assessment; Educational psychology.
1. INTRODUCTION
Our digital society is full of objects driven by software [1]. We
are living, more and more, embedded in a computer-based world.
Given this current reality, it is becoming indispensable to handle
the language of computers to participate fully and effectively in
the digital ecosystem that surrounds us. So it is not surprising that
the term ‘code-literacy’ has been recently coined to define the
process of teaching and learning to read-write with computer
programming languages [2, 3, 4]. Thus, we consider that a person
is code-literate when he/she is able to read and write in the
language of computers, and to think computationally [5]. If code-
literacy refers ultimately to an emerging read-write practice,
computational thinking (CT) refers to the underlying problem-
solving (only cognitive?) process that supports it. In other words,
computer programming is the fundamental way that enables CT to
come alive [6], although CT can be projected on different kinds of
problems that may not involve directly programming tasks [7].
In this context, CT is becoming considered in many countries as a
key set of problem-skills that must be developed by the new
generations of students. However, there is still little consensus
about a formal definition of CT [8, 9], an alarming gap about how
to measure and assess CT [9, 10]; and there are disagreements on
how it should be integrated in educational curricula [6].
Furthermore, from a psychometric approach, CT is not a well-
established psychological construct yet, as its nomological
network [11] has not been fully defined; that is, the correlations
between CT and other psychological constructs, whether (a)
cognitive or (b) non-cognitive, have not been completely reported
by the scientific community.
Regarding to (a), a very recent paper describes the relations
between CT and other cognitive variables [12]. These authors
report statistically significant correlations at least moderately
intense between CT and spatial ability (r = 0.44), reasoning ability
(r = 0.44), and problem-solving ability (r = 0.67). These results
empirically corroborate the conceptualization of CT as mainly a
problem-solving ability, linked with g or fluid intelligence; this
fact that had been theoretically stated by many other authors in
previous years [6, 7, 9, 13, 14].
But regarding to (b), as far as we know, no empirical research has
been conducted so far studying the correlations between CT and
non-cognitive variables. Thus, in this paper we specifically
attempt to answer the following research question:
RQ: Does computational thinking correlate with personality?
This question is plausible since it does exist prior evidence of
relationship between cognitive and personality variables, as we
describe in subsections 1.1 and 1.2. Besides, answering the
research question, even tentatively, is relevant as it may contribute
to define the nomological network of CT and consolidate its
consistency as an emerging psychological construct. Finally, the
answer may help to understand better which dimensions of the
human personality underlie CT, in order to promote and optimize
its development on educational settings.
1.1 Computational thinking
In her foundational article, ten years ago, Jeannette Wing defined
that CT “involves solving problems, designing systems, and
understanding human behavior, by drawing on the concepts
fundamental to computer science” [13]. Thus, we could say that
CT’s essence is thinking like a computer scientist when
confronted with a problem. But this first generic definition has
been revised over the last few years, still not reaching an
agreement [9, 10]. So, in 2011 Wing clarified: CT “is the thought
processes involved in formulating problems and their solutions so
that the solutions are represented in a form that can be effectively
carried out by an information-processing agent” [15]. One year
later, this definition is simplified by Aho, who conceptualizes CT
as the thought processes involved in formulating problems so
“their solutions can be represented as computational steps and
algorithms” [16].
But, from a psychometric approach, generic definitions (such as
those quoted above) are not enough. Operational definitions of the
psychological construct are needed to enable and guide its
development, measurement and assessment. In this direction, the
Computer Science Teachers Association (CSTA) and the
International Society for Technology in Education (ISTE) stated
in 2011 an operational definition of computational thinking that
provides a framework and common vocabulary for Computer
Science K-12 educators: CT is a “problem-solving process that
includes (but is not limited to) the following characteristics:
formulating problems in a way that enables us to use a computer
and other tools to help solve them; logically organizing and
analyzing data; representing data through abstractions such as
models and simulations; automating solutions through algorithmic
thinking (a series of ordered steps); identifying, analyzing, and
implementing possible solutions with the goal of achieving the
most efficient and effective combination of steps and resources;
generalizing and transferring this problem solving process to a
wide variety of problems” [17].
Moreover, the aforementioned operational definition continues
saying that “these (problem-solving) skills are supported and
enhanced by a number of dispositions or attitudes that are
essential dimensions of CT. These dispositions or attitudes
include:
Confidence in dealing with complexity.
Persistence in working with difficult problems.
Tolerance for ambiguity.
The ability to deal with open ended problems.
The ability to communicate and work with others to
achieve a common goal or solution” [17].
In other words, it is accepted, in a broad sense, the existence of a
non-cognitive side of CT, although it has not been empirically
confirmed yet.
1.2 Big Five model of personality
In the last 25 years, an impressive body of research has
accumulated supporting the validity of a five-factor structure to
describe human personality (the so-called ‘Big Five’) [18, 19].
The Big Five model, whose validity has been also demonstrated in
late childhood and early adolescence [20], states the following
labels for these five dimensions:
Conscientiousness (C): it refers to aspects such as
autonomy, dependability, orderliness, precision,
persistence, and the fulfilling of commitments.
Openness to Experience (O), also called Intellect: it
refers to self-reported intellect, especially in the school
domain, and broadness or narrowness of cultural
interests, self-reported fantasy/creativity, and interest in
other people.
Extraversion (E), also called Energy: it refers to aspects
such as sociability, activity, enthusiasm, assertiveness,
and self-confidence.
Agreeableness (A): it refers to aspects such as concern
and sensitivity towards others and their needs, tendency
to cooperation.
Neuroticism (N), also called Emotional Instability: it
refers to feelings of anxiety, depression, discontent,
irritability, and anger.
Examples of items aimed at assessing each of the five factors are
reported in paragraph 2.2.2, in which the Big Five Questionnaire-
Children version (BFQ-C) is described.
In summary, prior research on the Big Five model has
demonstrated that: Openness and Conscientiousness are positively
correlated with Academic Achievement [19, 20]; Openness is also
positively correlated with the Intelligence Quotient (IQ) [21];
Extraversion and Neuroticism are positively correlated with
Externalizing problematic behavior syndromes (hyperactivity,
transgressive/disruptive conduct, inattentiveness, and aggression)
[19]; and Neuroticism is also positively correlated with
Internalizing problematic behavior syndromes (depression,
anxiety, somatic complaints, and obsessiveness) [19].
If we intersect the dispositions or attitudes that underlie CT
according to the operational definition from CSTA & ISTE [17]
with the five personality dimensions stated from the Big Five
model [18, 19], the crosstab in Table 1 is obtained.
Table 1. Crosstab intersecting CT dispositions/attitudes with
the Big Five model of personality.
CT dispositions or attitudes
Big Five model
C
O
E
A
N
Confidence in dealing with complexity
-
*
/
-
-
Persistence in working with difficult
problems
*
-
-
-
-
Tolerance for ambiguity
/
*
-
-
-
The ability to deal with open ended
problems
-
*
-
-
-
The ability to communicate and work with
others to achieve a common goal
-
-
*
/
-
*: Yes; /: Partly; -: No.
Hence, it is expected that CT will correlate positively with
Openness (O); and, in a lesser extent, with Conscientiousness (C)
and with Extraversion (E). In this paper an empirical study aimed
at verifying these hypotheses is conducted.
2. Method
Following the recommendations for educational research [22], our
research method is reported in the following subsections:
participants (sample utilized in the research); instruments
(assessments tools administered to the sample); and procedure
(list of steps/actions followed so our research can be replicated).
2.1 Participants
The sample is composed by almost one hundred Spanish students
(n = 99) enrolled in classrooms from 5th to 10th grade (that is,
from 10/11 to 15/16 years old). In terms of gender, the sample is
balanced: 49 boys (49.5%) vs. 50 girls (50.5%). The distribution
of the subjects by gender and grade is shown in Table 2.
Table 2. Distribution of the sample (n = 99) by gender and
grade.
Grade
Total
5th & 6th
7th & 8th
9th & 10th
Gender
Boys
Count
22
15
12
49
% of Total
22.2%
15.2%
12.1%
49.5%
Girls
Count
24
13
13
50
% of Total
24.2%
13.1%
13.1%
50.5%
Total
Count
46
28
25
99
% of Total
46.5%
28.3%
25.3%
100.0%
All students of the sample performed the following assessment
instruments: the Computational Thinking Test (CTt), and the Big
Five Questionnaire-Children version (BFQ-C); which are
described below.
2.2 Instruments
2.2.1 Computational Thinking Test (CTt)
The Computational Thinking Test
1
(CTt) is a multiple-choice
instrument composed by 28 items, which are administered on-line
(via non-mobile or mobile electronic devices) in a maximum time
of 45 minutes. Each item of the CTt is designed and characterized
according to the following five dimensions [12, 23, 24]:
Computational concept addressed: each item
addresses one or more of the following seven
computational concepts, ordered in increasing difficulty:
Basic directions and sequences (4 items); Loops–repeat
times (4 items); Loops–repeat until (4 items); If–simple
conditional (4 items); If/else–complex conditional (4
items); While conditional (4 items); Simple functions (4
items). These ‘computational concepts’ are aligned with
the CSTA Computer Science Standards for 7th and 8th
grade [25].
Environment-Interface of the item: CTt items are
presented in any of the following two environments-
interfaces: ‘The Maze’ (23 items) or ‘The Canvas’ (5
items).
1
Available at http://goo.gl/IYEKMB (Spanish version). Other
forms and versions of CTt are available, free of charge, only for
research purposes, from the first author.
Style of answers: in each item, responses may be
presented in any of these two styles: Visual arrows (8
items) or Visual blocks (20 items).
Existence or non-existence of nesting: depending on
whether the solution of an item involves a script with
(19 items) or without (9 items) nesting computational
concepts (a concept embedded in another of a higher
hierarchy level).
Required task: depending on which of the following
cognitive tasks is required for solving the item:
Sequencing: the student must sequence, stating in an
orderly manner, a set of commands (14 items);
Completion: the student must complete an incomplete
given set of commands (9 items); Debugging: the
student must debug an incorrect given set of commands
(5 items).
Examples of CTt items translated into English are shown in
Figures 1, 2, 3 and 4; with their specifications detailed below.
Figure 1. CTt, item 6: loops–repeat times; ‘The Maze’; visual
arrows; no-nesting; completion.
Figure 2. CTt, item 8: loops-repeat times; ‘The Maze’; visual
blocks; yes-nesting; sequencing.
Figure 3. CTt, item 19: loops-repeat until + if/else conditional;
‘The Maze’; visual blocks; yes-nesting; debugging.
Figure 4. CTt, item 26: loops-repeat times + simple functions;
‘The Canvas’; visual blocks; yes-nesting; completion.
The design guidelines of the CTt and its content validation have
been already reported [23], as well as its descriptive statistics,
reliability (α ≈ 0.80) and criterion validity [12]. In summary, we
can affirm that the CTt is a reliable and valid test for assessing
computational thinking in students from 10 to 16 years. However
some of its limitations have been pointed out: e.g., CTt is heavily
focused on ‘computational concepts’ [14] and only provides a
static-summative assessment, which should be complemented
with other tools designed from a formative assessment perspective
such as Dr. Scratch
2
[26, 27].
2.2.2 Big Five Questionnaire-Children version
(BFQ-C)
The Big Five Questionnaire-Children version (BFQ-C) [19] is an
adaptation for child and adolescent population (8-15 years old)
derived from the original ‘adult’ BFQ [28], and is aimed at
assessing the personality of the subject within the Big Five model.
The BFQ-C is a questionnaire without time limit and composed
by 65 items; for each of them, the child rates the occurrence of the
behavior reported in the item using a 5-point Likert scale ranging
from 1 (=Almost never) to 5 (=Almost always). Below, some item
examples relative to each of the five personality factors are given:
Conscientiousness (C): “I work hard and with pleasure”;
“I engage myself in the things I do”; “During class-time
I am concentrated on the things I do”; “When I finish
my homework, I check it many times to see if I did it
correctly”; “I respect the rules and the order”; “If I take
an engagement I keep it”; “My room is in order”;
“When I start to do something I have to finish it at all
costs”; “I like to keep all my school things in a great
order”; “I play only when I finished my homework”; “It
is unlikely that I divert my attention”; “I do my own
duty”.
Openness to Experience (O): “I know many things”; “I
have a great deal of fantasy”; “I easily learn what I
study at school”; “When the teacher asks questions I am
able to answer correctly”; “I like to read books”; “When
the teacher explains something I understand
immediately”; “I like scientific TV shows”; “I like to
watch TV news, and to know what happens in the
world”; “I am able to create new games and
entertainments”; “I am able to solve mathematics
problems”; “I like to know and to learn new things”; “I
would like very much to travel and to know the habits
of other countries”.
2
http://drscratch.org/
Extraversion (E): “I like to meet with other people”; “I
like to compete with others”; “I like to move and to do a
great deal of activity”; “I like to be with others”; “I can
easily say to others what I think”; “I say what I think”;
“I do something not to get bored”; “I like to talk with
others”; “I am able to convince someone of what I
think”; “When I speak, the others listen to me and do
what I say”; “I like to joke”; “I easily make friends”.
Agreeableness (A): “I share my things with other
people”; “I behave correctly and honestly with others”;
“I understand when others need my help”; “I like to give
gifts”; “If someone commits an injustice to me, I forgive
her/him”; “I treat my peers with affection”; “I behave
with others with great kindness”; “I am polite when I
talk with others”; “If a classmate has some difficulty I
help her/him”; “I trust in others”; “I treat kindly also
persons who I dislike”; “I think other people are good
and honest”; “I let other people use my things”.
Neuroticism (N): “I get nervous for silly things”; “I am
in a bad mood”; “I argue with others with excitement”;
“I easily get angry”; “I quarrel with others”; “I easily
get offended”; “I am sad”; “I am not patient”; “I easily
lose my calm”; “I do things with agitation”; “I weep”; “I
worry about silly things”.
In our research, the Spanish version [29] of the BFQ-C was
administered as a ‘self-report’ form (the students answer the items
referring to themselves). The technical manual reports good
reliability for all the factors (α > 0.80), and statistically significant
positive correlations between Openness (O) and Academic
Achievement (r= 0.51), and between Conscientiousness (C) and
Academic Achievement (r= 0.13) [29].
2.3 Procedure
Participating subjects in our research were enrolled in the elective
subject of Computer Science, which is held twice a week (one
hour each). Typically, the CTt was administered during the first of
the two weekly classes, and the BFQ-C during the second weekly
class. None of the subjects had prior computer programming
formal experience when the CTt was administered.
For the CTt collective administration, the Computer Science
teacher followed the instructions which were sent by email the
week before, containing the URL to access the on-line test. The
student’s direct answers to the CTt items were stored in the
Google Drive database linked with the instrument, which was
subsequently downloaded as an Excel .xls file.
For the collective administration of the BFQ-C, students were
previously signed in the on-line platform from the publishing
house
3
holder of this questionnaire’s commercial rights. Come the
administration day, the subjects logged in the platform and
performed the questionnaire. Afterwards, from our administrator
profile, we could download the subjects’ results as an Excel .xls
file.
Finally, all .xls files generated during data collection were
exported to a single .sav file, which constitutes the data matrix
under analysis with the SPSS software (version 22). Results
shown below arise from this analysis.
3
http://www.e-teaediciones.com/
3. RESULTS
Correlations (expressed as Pearson’s r) between scores from CTt
and the five personality factors assessed through BFQ-C are
shown in Table 3.
Table 3. Correlations CTt*BFQ-C (n = 99)
BFQ-C
C
O
E
A
N
CTt
0.267**
0.407**
0.304**
0.133
0.092
n = 99; **p < 0.01
As it can be seen, the CTt has a positive statistically significant
correlation (p < 0.01) with three of the five personality factors
assessed through BFQ-C: moderately intense with the Openness
(O) factor, and slightly intense with the Extraversion (E) and
Conscientiousness (C) factors. There is no statistically significant
correlation between CTt and the Agreeableness (A) and
Neuroticism (N) factors. Corresponding scatter plots and
coefficients of determination (R2) are shown below (Figures 5, 6,
7, 8 and 9).
Figure 5. Scatter plot between CTt and Conscientiousness (C)
Figure 6. Scatter plot between CTt and Openness (O)
Figure 7. Scatter plot between CTt and Extraversion (E)
Figure 8. Scatter plot between CTt and Agreeableness (A)
Figure 9. Scatter plot between CTt and Neuroticism (N)
4. DISCUSSION
In summary, our results are partially consistent with the literature
about the links between cognitive and personality variables, and
fit notably with our expectations after intersecting the dispositions
or attitudes that underlie CT according to the operational
definition from CSTA & ISTE [17] with the five personality
dimensions stated from the Big Five model [18, 19] (Table 1).
On the one hand, the positive and moderately intense correlation
found between CT and the Openness (O) factor is consistent with
the prior research, which states that the ‘O’ factor is the one
strongest related with cognitive variables such as intelligence or
academic achievement [19, 20, 21]. Finding this correlation value
was expected, as there are several attitudes underlying CT linked
with Openness (e.g. “The ability to deal with open ended
problems”). Moreover, the positive but in a lesser extent slight
correlation between CT and the Conscientiousness (C) factor is
also consistent with the body of research, which links lightly the
‘C’ factor with the aforementioned cognitive variables. This was
expected too, as there are also some dispositions supporting CT
related with Conscientiousness (e.g. “Persistence in working with
difficult problems”).
On the other hand, and following an analogous discussion thread,
the absence of correlation between CT and the Agreeableness (A)
and Neuroticism (N) factors, is also consistent with literature; and
was expected according to Table 1 too, as there are no attitudes
enhancing CT related to these factors (except “The ability to
communicate and work with others to achieve a common goal or
solution”, which can be partly linked with the ‘A’ factor).
Finally, and this is the point which requires deeper discussion, the
positive and slight (but considerable r = 0.30) correlation between
CT and the Extraversion (E) factor seems surprising. Although the
‘E’ factor intersects with some CT attitudes or dispositions from
the CSTA & ISTE operational definition (Table 1), as far as we
know previous research had never demonstrated any positive
correlation between the ‘E’ and cognitive variables (as CT is
mainly supposed to be). Actually, some evidence of slight
negative correlation exists in the literature between the ‘E’ factor
and, for example, Academic Achievement (r = -0.13) [19]. All of
the above led us to speculate that Extraversion (also called
Energy) might be a specific personality trait of top computational
thinkers. We find three types of evidences that may support this
argumentation.
Firstly, there are some emerging and comprehensive CT
assessment frameworks that take into account leadership
and collaboration skills [30], or assertiveness and
effective communication skills [31] (which are at the
core of ‘E’ factor), as important ingredients of CT.
Secondly, we have some recent qualitative evidences
[32], in which teachers report the unexpected brilliant
performance/behavior of students usually
disruptive/inattentive, when faced with computer
programming experiences (e.g., Code.org courses
4
). In
other words, students with some Externalizing problems
(hyperactivity, disruptive conduct, inattentiveness)
linked with the ‘E’ factor seem to respond especially
well to programming tasks. This may explain the
specific relation ‘E’*CT.
4
https://studio.code.org/s/20-hour
Thirdly, speculating that top computational thinkers
might be precursors of future top developers in Open
Source Software (OSS) communities, we collect
evidence about which personality traits are specific
from the latter. Thus, Rigby et al. [33] use the
dimensions of a psychometrically-based linguistic
analysis tool to study the Big Five personality traits in
the Apache httpd server developer mailing list. They
found that of the four most active developers in the two
versions of Apache under study, two of them show
similar personalities that differed from the other
developers in Extraversion and Openness (the third top
developer had similar personality traits than the rest,
while the last one was not consistently associated with
anyone else) [33]. Moreover, Bazelli et al. analyze the
Q&A website StackOverflow for personality traits of
participants [34]: using textual analysis of posts, they
found that authors with higher reputation are more
extroverted compared to those with medium and lower
reputation.
5. CONCLUSIONS AND FURTHER
RESEARCH
In this paper we have provided empirical evidence of the
correlations between CT and the five factors of personality from
the ‘Big Five’ model. We have found expected positive
correlations with Openness (r = 0.41) and Conscientiousness (r =
0.27), and unexpected positive correlation with Extraversion (r =
0.30). These results corroborate the idea that, although CT is
mainly a cognitive psychological construct close to problem-
solving ability [12], there is also a complementary non-cognitive
side of CT.
From this starting point, further research can be conducted in the
following directions:
A multiple regression model of the CTt onto the BFQ-C
factors can be constructed; and afterwards it can be
compared/integrated with the complementary model
reported in [12], in which a multiple regression of the
CTt onto the Primary Mental Abilities (verbal, spatial,
reasoning, and numerical) battery is performed. Once
done, it will be possible to depict a quite complete
nomological network of CT including both cognitive
and non-cognitive correlations, and their overlapping
[35].
Research aimed at studying if reinforcement of
behaviors related to Openness, Conscientiousness, and
Extraversion, can actually foster CT learning and
development in educational settings.
Finally, case studies on the effect of computer
programming tasks over students with ADHD
(Attention-Deficit Hyperactivity Disorder) and/or DBD
(Disruptive Behavior Disorders) may be conducted.
6. REFERENCES
[1] Manovich, L. (2013). Software Takes Command. New York:
Bloomsbury.
[2] Prensky, M. (2008, January 13). Programming Is the New
Literacy [Blog post]. Retrieved from
http://www.edutopia.org/literacy-computer-programming
[3] Rushkoff, D. (2010). Program or be programmed. New
York: OR Books.
[4] Rushkoff, D. (2012, November 13). Code Literacy: A 21st-
Century Requirement [Blog post]. Retrieved from
http://www.edutopia.org/blog/code-literacy-21st-century-
requirement-douglas-rushkoff
[5] Román-González, M. (2014). Aprender a programar ‘apps’
como enriquecimiento curricular en alumnado de alta
capacidad. Bordón. Revista de Pedagogía, 66(4), 135-155.
http://dx.doi.org/10.13042/Bordon.2014.66401
[6] Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and
learning of computational thinking through programming:
What is next for K-12? Computers in Human Behavior, 41,
51-61. http://dx.doi.org/10.1016/j.chb.2014.09.012
[7] Wing, J. M. (2008). Computational thinking and thinking
about computing. Philosophical Transactions. Series A,
Mathematical, Physical, and Engineering Sciences,
366(1881), 3717-3725.
http://dx.doi.org/10.1098/rsta.2008.0118
[8] Gouws, L. A., Bradshaw, K., & Wentworth, P. (2013).
Computational thinking in educational activities: An
evaluation of the educational game light-bot. Proceedings of
the 18th ACM Conference on Innovation and Technology in
Computer Science Education, 10-15.
http://dx.doi.org/10.1145/2462476.2466518
[9] Kalelioğlu, F., Gülbahar, Y., & Kukul, V. (2016). A
Framework for Computational Thinking Based on a
Systematic Research Review. Baltic Journal of Modern
Computing, 4(3), 583-596. Retrieved from
http://www.bjmc.lu.lv/fileadmin/user_upload/lu_portal/proje
kti/bjmc/Contents/4_3_15_Kalelioglu.pdf
[10] Grover, S., & Pea, R. (2013). Computational thinking in K–
12: A review of the state of the field. Educational
Researcher, 42(1), 38-43.
http://dx.doi.org/10.3102/0013189X12463051
[11] Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in
psychological tests. Psychological Bulletin, 52(4), 281-302.
http://dx.doi.org/10.1037/h0040957
[12] Román-González, M., Pérez-González, J.C., & Jiménez-
Fernández, C. (2016). Which cognitive abilities underlie
computational thinking? Criterion validity of the
Computational Thinking Test. Computers in Human
Behavior, in press.
http://dx.doi.org/10.1016/j.chb.2016.08.047
[13] Wing, J. M. (2006). Computational
thinking. Communications of the ACM, 49(3), 33-35.
http://dx.doi.org/10.1145/1118178.1118215
[14] Brennan, K., & Resnick, M. (2012). New frameworks for
studying and assessing the development of computational
thinking. Proceedings of the 2012 Annual Meeting of the
American Educational Research Association (Vancouver:
Canada). Retrieved from
http://scratched.gse.harvard.edu/ct/files/AERA2012.pdf
[15] Wing, J. M. (2011). Research Notebook: Computational
Thinking--What and Why? The Link. The magazine of the
Carnegie Mellon University School of Computer Science.
Retrieved from http://www.cs.cmu.edu/link/research-
notebook-computational-thinking-what-and-why
[16] Aho, A. V. (2012). Computation and computational
thinking. The Computer Journal, 55(7), 832-835.
http://dx.doi.org/10.1093/comjnl/bxs074
[17] CSTA & ISTE (2011). Operational Definition of
Computational Thinking for K–12 Education. Retrieved from
http://csta.acm.org/Curriculum/sub/CurrFiles/CompThinking
Flyer.pdf
[18] Goldberg, L. R. (1990). An alternative “Description of
personality”: the Big Five factor structure. Journal of
Personality and Social Psychology, 59, 1216–1229.
http://dx.doi.org/10.1037/0022-3514.59.6.1216
[19] Barbaranelli, C., Caprara, G. V., Rabasca, A., & Pastorelli,
C. (2003). A questionnaire for measuring the Big Five in late
childhood. Personality and Individual Differences, 34(4),
645-664. http://dx.doi.org/10.1016/S0191-8869(02)00051-X
[20] Mervielde, I., Buyst, V., & De Fruyt, F. (1995). The validity
of the Big-Five as a model for teachers’ ratings of individual
differences among children aged 4–12 years. Personality and
Individual Differences, 18(4), 525-534.
http://dx.doi.org/10.1016/0191-8869(94)00175-R
[21] John, O., Caspi, A., Robins, R., Moffitt, T., & Stouthamer-
Loeber, M. (1994). The “Little Five”: Exploring the
Nomological Network of the Five-Factor Model of
Personality in Adolescent Boys. Child Development, 65(1),
160-178. Retrieved from
http://www.jstor.org/stable/1131373
[22] Cohen, L., Manion, L., & Morrison, K. (2007). Research
methods in education. New York: Routledge.
[23] Román-González, M. (2015). Computational Thinking Test:
Design Guidelines and Content Validation. 7th annual
International Conference on Education and New Learning
Technologies (Barcelona: Spain).
http://dx.doi.org/10.13140/RG.2.1.4203.4329
[24] Román-González, M., Pérez-González, J.C., & Jiménez-
Fernández, C. (2015). Test de Pensamiento Computacional:
diseño y psicometría general [Computational Thinking Test:
design & general psychometry]. III Congreso Internacional
sobre Aprendizaje, Innovación y Competitividad, CINAIC’
2015 (Madrid: Spain).
http://dx.doi.org/10.13140/RG.2.1.3056.5521
[25] CSTA (2011). K–12 Computer Science Standards. Retrieved
from http://csta.acm.org/Curriculum/sub/CurrFiles/CSTA_K-
12_CSS.pdf
[26] Moreno-León, J., & Robles, G. (2015). Dr. Scratch: a Web
Tool to Automatically Evaluate Scratch Projects.
Proceedings of the 10th Workshop in Primary and Secondary
Computing Education, 132-133.
http://dx.doi.org/10.1145/2818314.2818338
[27] Moreno-León, J., Robles, G., & Román-González, M.
(2015). Dr. Scratch: Automatic Analysis of Scratch Projects
to Assess and Foster Computational Thinking. RED, Revista
de Educación a Distancia, 46. Retrieved from
http://www.um.es/ead/red/46/moreno_robles.pdf
[28] Caprara, G. V., Barbaranelli, C., Borgogni, L., & Perugini,
M. (1993). The “Big five questionnaire”: A new
questionnaire to assess the five factor model. Personality and
Individual Differences, 15(3), 281-288.
http://dx.doi.org/10.1016/0191-8869(93)90218-R
[29] Barbaranelli, C., Caprara, G. V., & Rabasca, A. (2013).
BFQ-NA Cuestionario “Big Five” de Personalidad para
Niños y Adolescentes [Technical manual]. Madrid: TEA
Ediciones.
[30] Snow, E. (2014). Assessing Computational Thinking
[Presentation]. Retrieved from
http://pact.sri.com/downloads/AssessmentBreakout.pdf
[31] Grover, S. (2015). “Systems of Assessments” for Deeper
Learning of Computational Thinking in K-12. Proceedings of
the 2015 Annual Meeting of the American Educational
Research Association (Chicago, USA). Retrieved from
http://web.stanford.edu/~shuchig/docs/AERA2015-Grover-
Systems%20of%20Assessments%20for%20Deeper%20Lear
ning%20of%20Computational%20Thinking%20in%20K-
12.pdf
[32] Román-González, M. (2016). Codigoalfabetización y
pensamiento computacional en educación primaria y
secundaria: validación de un instrumento y evaluación de
programas [Doctoral thesis]. Madrid: UNED.
[33] Rigby, P. C., & Hassan, A. E. (2007). What can OSS mailing
lists tell us? A preliminary psychometric text analysis of the
Apache developer mailing list. Proceedings of the Fourth
International Workshop on Mining Software
Repositories, 23. http://dx.doi.org/10.1109/MSR.2007.35
[34] Bazelli, B., Hindle, A., & Stroulia, E. (2013). On the
personality traits of StackOverflow users. Proceedings of the
19th International Conference on Software Maintenance,
460-463. Retrieved from
http://webdocs.cs.ualberta.ca/~hindle1/2014/bazelli-
personality-draft.pdf
[35] Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence,
personality, and interests: Evidence for overlapping
traits. Psychological Bulletin, 121(2), 219-245.
http://dx.doi.org/10.1037/0033-2909.121.2.219