Does Self-Efficacy Correlate with Positive Emotion and Academic Performance in Collaborative Learning?

Abstract

This full research paper studies the correlation of self-efficacy in computer science as well as learning and social skills with students' academic performance and their emotions in collaborative learning environments. Self-efficacy is an essential part of social cognitive theory and provides the foundation for analyzing human thoughts, motivations, and actions. Studies show that students' successful performance and accomplishment are directly affected by the level of self-efficacy. Therefore, analyzing self-efficacy in engineering education is important since it can impact the learning process in academic settings as well as provide a metric to track for improvement. Social cognitive theories also emphasize that students' interaction with each other affects their learning process and how they perform in educational settings. In previous work [5], we analyzed students' conversations in low-stake teams in an introductory programming course (CS1) and observed a strong positive correlation between students' positive emotions while interacting with each other with their performance in the course. In this study, we focus on the correlation of self-efficacy with learner's emotion and performance. We measure students' self-efficacy with a standard instrument called "Student Attitudes Toward STEM (S-STEM) Survey". For this purpose, we asked the participants to self-report on a 5-point Likert-scaled survey including 20 questions. These 20 questions are grouped into 2 main categories of computer science and learning/social skills. Students' emotions were extracted from their speeches in teams by applying natural language processing (NLP) methods. The result of data analysis shows a statistically significant correlation between overall self-efficacy and performance in the course and positive emotions during the teamwork. We further investigate which category of self-efficacy questions most correlate with students' performance. The result shows self-efficacy in interpersonal skills and learning ability most impact students' performance.

Full text

Does Self-Efficacy Correlate with Positive Emotion
and Academic Performance in Collaborative
Learning?
Nasrin Dehbozorgi
Department of Software Engineering
Kennesaw State University
Marietta, GA, USA
dnasrin@kennesaw.edu
Mary Lou Maher
Department of Software Information Systems
UNC Charlotte
Charlotte, NC, USA
mmaher@uncc.edu
Mohsen Dorodchi
Department of Computer Science
UNC Charlotte
Charlotte, NC, USA
mohsen.dorodchi@uncc.edu
Abstract— This full research paper studies the correlation of
self-efficacy in computer science as well as learning and social
skills with students’ academic performance and their emotions
in collaborative learning environments. Self-efficacy is an
essential part of social cognitive theory and provides the
foundation for analyzing human thoughts, motivations, and
actions. Studies show that students’ successful performance and
accomplishment are directly affected by the level of self-efficacy.
Therefore, analyzing self-efficacy in engineering education is
important since it can impact the learning process in academic
settings as well as provide a metric to track for improvement.
Social cognitive theories also emphasize that students’
interaction with each other affects their learning process and
how they perform in educational settings. In previous work [5],
we analyzed students’ conversations in low-stake teams in an
introductory programming course (CS1) and observed a strong
positive correlation between students’ positive emotions while
interacting with each other with their performance in the
course. In this study, we focus on the correlation of self-efficacy
with learner’s emotion and performance. We measure students’
self-efficacy with a standard instrument called “Student
Attitudes Toward STEM (S-STEM) Survey”. For this purpose,
we asked the participants to self-report on a 5-point Likert-
scaled survey including 20 questions. These 20 questions are
grouped into 2 main categories of computer science and
learning/social skills. Students’ emotions were extracted from
their speeches in teams by applying natural language processing
(NLP) methods. The result of data analysis shows a statistically
significant correlation between overall self-efficacy and
performance in the course and positive emotions during the
teamwork. We further investigate which category of self-
efficacy questions most correlate with students’ performance.
The result shows self-efficacy in interpersonal skills and
learning ability most impact students’ performance.
Keywords—self-efficacy, social skill, computer science,
emotion mining, sentiment analysis, NLP, collaborative active
learning
I. INTRODUCTION
Attitude is a complex subject that has several dimensions
such as affect, cognition, and behavior [1, 2]. It is defined as a
tendency to have certain beliefs and feelings about a given
context and involves behavioral aspects to accomplish a task
and achieve the goal [2]. Attitude is a key element in students’
learning in the academic setting and is observable through
students’ behavior in class and how they engage in class
activities and teams [2]. Students’ attitude impacts their
success in the given domain, as well as their interpersonal
relationships [1,3,4].
Emotion and affective states are important aspects of
attitude. According to research, positive emotions like joy,
happiness, and satisfaction about the given subject positively
influence students’ learning experience [5] while emotional
obstacles such as anger, anxiety, etc. can hinder their
cognition process [6].
Self-efficacy is one other construct of the attitudinal
domain which is defined as how individuals judge and
perceive their ability to perform a specific task [2,7]. Self-
efficacy is the perception that one can successfully accomplish
a given task and the motivation to apply learned skills [7]. It
has its roots in self-regulation and is an integral part of social
cognitive theory [33, 7, 8]. It is believed that self-efficacy has
a positive impact on students’ academic performance and has
drawn the attention of researchers in engineering education for
the past several years [7, 8, 9]. Identifying the factors that
contribute to self-efficacy can help educators apply required
interventions to improve students’ learning experiences [7].
According to research students,’ academic success and
accomplishment are results of higher self-efficacy, which in
turn boosts self-efficacy [7, 8].
Different factors like various background experiences,
interpersonal skills, as well as physical and emotional states
can impact self-efficacy, which makes it a dynamic attribute
that can change over time [7]. This dynamic aspect of self-
efficacy makes it difficult to measure and analyze, especially
in the educational setting.
While the tone of existing research emphasizes the key
role of self-efficacy in students’ academic success [7, 8], some
researchers witness a lack of correlation between self-efficacy
and academic performance in some populations of students
and in different situations [10]. Our observation from the
existing literature is that the correlation of self-efficacy and
performance depends on different factors, like the background
2021 IEEE Frontiers in Education Conference (FIE) | 978-1-6654-3851-3/21/$31.00 ©2021 IEEE | DOI: 10.1109/FIE49875.2021.9637471
Authorized licensed use limited to: Kennesaw State University. Downloaded on April 02,2024 at 17:21:10 UTC from IEEE Xplore. Restrictions apply.

of students, their age, major, and the learning environment and
class setting.
In this study, we investigate the correlation of self-efficacy
with emotion and students’ performance in an introductory
computer science class (CS1) which is conducted in a
collaborative active learning setting. We further analyze
which components of self-efficacy most correlate with
students’ performance. Our finding shows that students’ self-
efficacy in their interpersonal and learning skills is strongly
correlated with how they perform in the course. Our analysis
also shows a positive correlation between self-efficacy and
positive emotion in this study group.
The rest of this paper is organized as follows; we review
the literature on attitude constructs and their impact on
students’ performance, next we present our research
methodology, the study design, and data collection protocol,
we present the result of data analysis and test the hypotheses
followed by the conclusion and plan for the future work.
II. BACKGROUND
The impact of self-efficacy and affective states on
students’ performance makes them important concepts to
focus on in engineering education. Such concepts help
instructors to identify the students who experience emotional
difficulties or have lower levels of self-conception during the
early stages of the course and to provide them appropriate
feedback [11].
The relationship between self-efficacy and academic
performance at the college level has been widely studied by
researchers in the field [12]. In [13] the authors found that, in
the context of a CS1 course, in particular, self-efficacy
affected students’ performance in learning programming and
that this skillset can be used to predict students’ performance
[9, 8]. In engineering education research, self-efficacy is
considered as a bridge to connect the past experience with the
future performance outcome [12]. This applies to both high
school and college-level students [21].
In addition to the impact of self-efficacy on students’
performance, research shows self-efficacy also impacts
students’ major and future career choices [22, 20]. It is shown
that higher-level self-efficacy and social support play a big
role in orienting students to computing careers [23].
Literature suggests that other factors like gender [18], age
[14,15,16], and prior experience [17,13,18] impact the
correlation between self-efficacy and performance in college-
level students. Furthermore, the combinations of self-efficacy
and gender result in different patterns in learning
programming [8]. For example, in [18] the researchers found
that female and male students with no prior background in
programming performed at the same level while the females
with prior experience had higher performance than males with
prior experience in programming. In another study [19] the
authors examined the correlation of self-efficacy and goal
orientation with performance. The result showed the
connection of self-efficacy with performance among female
students is different from that of male peers [19].
In another study, the authors analyze the correlation
among the triangle of computer self-efficacy, learning
performance, and learning engagement. They apply the
General Self-Efficacy Scale developed by Schwarzer, et.al.
[28] in adults who seek computing as their second job. The
result of their study shows that computer self-efficacy is
positively correlated with learning performance and learning
engagement. They also found that learning engagement is
positively related to learning performance [29].
Different methods and standard instruments have been
developed to measure self-efficacy in educational settings.
Some of the common ones are the “Student Attitudes Toward
STEM (S-STEM) Survey” [27], Motivated Strategies for
Learning Questionnaire (MSLQ) [25], and General Self-
Efficacy Scale (GSE) [24].
One of the most common approaches to measure self-
efficacy is having students fill out surveys such as the
Motivated Strategies for Learning Questionnaire (MSLQ)
self-report tool [8]. It is a common self-report tool to measure
student motivation and learning strategies in a given context
[8].
One of the studies that applied the Motivated Strategies for
Learning Questionnaire (MSLQ) [25] measured the self-
efficacy of 39 students in a CS1 class. The result of this study
showed a strong positive correlation between motivational
and learning strategy scales with students’ performance in the
course [25]. Again, in another study [26], the authors identify
MSLQ scores to be highly correlated with students’
performance.
“Student Attitudes Toward STEM (S-STEM) Survey”
[27] is a widely used instrument to measure the self-efficacy
of students in the STEM field. This tool asks participants
about their confidence level and attitude disposition about
STEM subjects and career areas as well as the required skills
for the 21st century. The result of this test can be applied by
administrators and program directors to adjust the curriculum
accordingly [35].
In this work, we adapt the S-STEM tool to measure
students’ self-efficacy in learning computer science as well as
their self-efficacy about social skills in a CS1 collaborative
active learning class. In particular, we study if students’ self-
efficacy about their learning and interpersonal skills correlate
with their performance and their positive emotions. In the
following section, we present our research hypotheses and the
methodology.
III. RESEARCH METHOD
The main goal of this study is to identify the correlation of
self-efficacy with positive emotions and performance. We
conduct the case study on the population of mixed female and
male students with diverse backgrounds in a programming
class (CS1).
In order to measure students’ self-efficacy, we apply the
Student Attitudes Toward STEM (S-STEM) tool. Students’
emotions are extracted by Natural Language Processing
(NLP) methods from their collaborative speech in low-stake
teams [5] in which the emphasis is on building communication
Authorized licensed use limited to: Kennesaw State University. Downloaded on April 02,2024 at 17:21:10 UTC from IEEE Xplore. Restrictions apply.

skills and learning from peers rather than a group grade for the
outcome. The students’ individual grade in the course is
considered as the performance metric. In this study, we want
to identify the association between self-efficacy and
performance as well as students’ emotional disposition in
teams. We further identify the self-efficacy factors that impact
students’ programming performance.
To answer the research question, we formulate the
following Null hypotheses:
H10. There is a correlation between students’ self-efficacy
and individual performance.
H20. There is a correlation between students’ self-efficacy
and positive sentiments (frequency and intensity of compound
values) in low-stake teams.
The research methodology is categorized into three main
phases: 1) identify a standard tool to measure self-efficacy, 2)
describe an emotion mining algorithm to measure the positive
sentiments, 3) present a study design for data collection. The
high-level visualization of the tools and methods we applied
for data collection from the beginning to the end of a semester
is presented in Fig. 1. We elaborate on these tools and methods
in the remainder of this section.
Fig. 1. Study design for data collection in one semester
A. Self-efficacy
For measuring students’ self-efficacy, we applied the
“Student Attitudes Toward STEM (S-STEM) Survey” which
is a standard self-report tool [27]. S-STEM is a five-point
Likert Scale tool that measures students’ self-efficacy in five
categories of ‘Math’, ‘Science’, ‘engineering and technology,
‘21st-century learning’, and ‘about yourself’. The category of
‘21st-century learning’ focuses on soft skills like collaboration
and social skills and the ‘about yourself’ section measures
how the individuals expect themselves to perform in the given
domain.
In this study, we focus on the ‘science’ (CS) and ‘the 21st-
century learning’ (L&S) components of the self-efficacy tool
to be applied in a CS1 class. The questions in each category
are presented in Table 1.
A self-efficacy test was conducted at the beginning of the
semester to measure the level of self-efficacy in students. The
response rate to this survey was 100% which enabled us to use
this as a metric and analyze its correlation with students’
performance and positive sentiments.
TABLE 1. S-STEM QUESTIONS [27]
ID
Description
CS1 I am sure of myself when I do computer science.
CS2 I would consider a career in computer science.
CS3 I expect to use computer science when I get out of College.
CS4 Knowing computer science will help me earn a living.
CS5 I will need computer science for my future work.
CS6 I know I can do well in computer science.
CS7 Computer science will be important to me in my life’s work.
CS8 I can handle most subjects well, but I cannot do a good job with
computer science.
CS9 I am sure I could do advanced work in computer science
L&S1 I am confident I can lead others to accomplish a goal.
L&S2 I am confident I can encourage others to do their best.
L&S3 I am confident I can produce high-quality work.
L&S4 I am confident I can respect the differences of my peers.
L&S5 I am confident I can help my peers.
L&S6 I am confident I can include others’ perspectives when making
decisions.
L&S7 I am confident I can make changes when things do not go as
planned.
L&S8 I am confident I can set my own learning goals.
L&S9 I am confident I can manage my time wisely when working on
my own.
L&S10 When I have many assignments, I can choose which ones need
to be done first.
L&S11 I am confident I can work well with students from different
backgrounds.
B. Sentiments
For operationalizing students’ positive emotions, we
recorded their speech while they worked in teams in class
during the semester. NLP methods (i.e., NLTK, VADER)
were adopted to extract the polarity of students’ sentiments.
The recording process in the classroom had some challenges,
the main one being the environmental noise level as all
crosstalk during class time. To improve the quality of recorded
audios for analysis we applied voice filtering and noise
reduction methods on them. Next, we cleaned the data by
assigning a unique ID to each speaker based on a voice
recognition mechanism.
The algorithm for text mining and sentiment analysis
from speech corpora is presented in our previous work [5].
This algorithm took the transcribed speech tokens as input and
classified the sentiments in three classes of positive, negative,
and neutral emotion as well as the compound value which is a
unique metric calculated based on the three classes of
emotion.
In order to analyze the positive sentiments, we measure
both the intensity and frequency of sentiments by applying
NLTK and VADER algorithms on the tokenized speech
vectors. The output feature vectors are sentiment classes of
positive, neutral, and negative as well as compound scores as
shown in Fig. 2.
Fig.2. The process of extracting positive sentiments (com>0) from speech
Authorized licensed use limited to: Kennesaw State University. Downloaded on April 02,2024 at 17:21:10 UTC from IEEE Xplore. Restrictions apply.

Here R is the total recorded speech dataset of each
participant during the semester and ST is speech segmentation
based on the initiation of talks, and V is the feature vector
consisting of four features of Pos, Neu, Neg, and Comp.
To extract the positive emotions, we considered the
vectors with compound values greater than zero (comp >0).
We normalize the intensity and frequency of positive
sentiments based on the number of speech segments (n) in
each dataset (R). Frequency refers to the number of vectors
that have positive compound values (comp>0) and intensity
refers to the actual value of compound scores in each vector.
Equations (1) and (2) shows how the mean frequency and
intensity values of positive sentiments are calculated [5]:
(1)
(2)
Where:
n= total number of vectors in each dataset
Vectors with positive comp_value= {1,2, …m}
The result presented in our previous study [5] showed a
strong positive correlation between students’ positive
emotions as they communicated in teams with their
performance in the course. In this study, we use the same
result of sentiment analysis to identify the correlation of
positive emotions with self-efficacy.
C. Data collection
We collected data from a CS1 active learning class with
63 students. From the total number of students, 48 students
participated in the study. The study design was such that
students were assigned to work in pairs during the class
activity. Using a collaborative active learning approach to
teaching this course, the class time was divided into three
parts: 1) a low-stake poll quiz from the previous content
followed by a Q/A discussion to resolve students’
misconceptions, 2) a mini-lecture on the prep work that
students were supposed to study before attending the class,
and 3) teamwork class activity. The class duration was 75
minutes for two sessions per week and about 40 minutes of
each session was dedicated to low-stake teamwork and class
activity. We recorded students’ conversations as they talked
in teams to solve given problems. The recording was done in
every session of the class, however, if one team member didn’t
attend a class we excluded that team from recording for that
session. This resulted in dropping the teams that missed more
than two sessions of recording for our analysis. As a result, we
could collect data from 28 students consistently throughout
the semester. Accordingly, we only considered the self-
efficacy and performance scores of these 28 students for data
analysis. In the following section, we present the result of the
data analysis.
IV. DATA ANALYSIS
In this study, students’ performance was evaluated in a
formative style based on three major assignments, three
lecture tests, and three lab tests, and multiple quizzes and class
activities during the semester. Each assessment had a certain
contribution to the final grade: Assignment 30%, Lecture test
30%, and Lab test 30%, and class activities 10%. Participants’
grade distribution in the course is presented in Fig. 3.
Fig. 3. Participants’ grade distribution in the course
As mentioned earlier we employed a standard tool called
the “Student Attitudes Toward STEM (S-STEM) Survey”
which is a standard self-report tool to measure students’ self-
efficacy. This is a five-point Likert Scale tool including 20
questions (selected for this study) with the answers ranging
from strongly disagree = 1 to strongly agree = 5. The theme of
the questions is grouped into two categories of Computer
science (CS), learning/social skills (L&S). The list of
questions is provided in Table 1.
The students were asked to self-report and provide
answers to these questions. Fig. 4 present distribution of
students’ answers to the 9 questions in the Science category
and Fig. 5 show students’ answers to the 11 questions in the
learning/social skills category.
Fig. 4. Distribution of students’ answers to the Science category in the S-
STEM tool
In order to test the null hypotheses, we apply the chi-
square test, and to measure the strength of association between
parameters (i.e. self-efficacy, emotion, and performance) we
adopt Spearman’s rank correlation coefficient method. The
first Null hypothesis states: there is a correlation between
students’ initial self-efficacy and individual performance.
Authorized licensed use limited to: Kennesaw State University. Downloaded on April 02,2024 at 17:21:10 UTC from IEEE Xplore. Restrictions apply.

Fig. 5. Distribution of students’ answers to the learning/social skills
category in the S-STEM tool
We use the chi-square test and measure the two-tailed p-
value with the confidence level of 0.05. The calculated p-value
is 0.61 which is higher than the confidence level of 0.05.
Therefore, the Null hypothesis cannot be rejected. We
conclude that there is a statistically significant correlation
between students’ self-efficacy and their performance.
After identifying the correlation, we measure the strengths
of the association between self-efficacy and performance.
Researchers such as Norman G. (2010) suggest that both
parametric and non-parametric methods can be applied to the
ordinal data and Likert-scaled data [30]. We applied
Spearman’s rank correlation coefficient, which is a
nonparametric (distribution-free) rank statistic method [31].
This method describes the relationship between two variables
based on a monotonic function, without making any
assumptions about the frequency distribution of the variables
[31]. The range of coefficient value (rs) is between -1 and 1.
The closer rs is to +1 or -1, means the two variables have a
stronger monotonic relationship. In Spearman’s rank
correlation coefficient, the value of rs determines the strength
of the correlation and is calculated based on Equation (3)[32].
(3)
where:
d = difference in ranks for variables
n = number of cases
Table 2. shows the interpretation of the rs value in
Spearman’s rank correlation coefficient [32]. The calculated
value of rs is 0.11 which denotes a positive yet very weak
association between the two variables of self-efficacy and
performance.
Next, we test the second Null hypothesis which states there
is a correlation between students’ self-efficacy and positive
sentiments in low-stake teams. In this step again like the
previous one, we test the hypothesis using the chi-square test
and measure the intensity of the relationship by applying
Spearman’s rank correlation coefficient.
The calculated p-value for the frequency of positive
emotions (vectors with comp>0) is 0.77 and the p-value for
the intensity of the positive sentiments is 0.75. For both
variables, the calculated p-value is higher than the confidence
level of 0.05, so the Null hypothesis is not rejected which
indicates there is a statistically significant correlation between
students’ self-efficacy and their positive sentiments.
TABLE 2. INTERPRETATION OF THE RS VALUE
rs value Interpretation
00
-
.19
very weak
.20
-
.39
weak
.40
-
.59
moderate
.60
-
.79
strong
.80
-
1.0
very strong
The Spearman’s Correlation Coefficient test shows the
result of rs = 0.06 for both the frequency and the intensity of
positive emotions which means the association between self-
efficacy and positive emotion (frequency and intensity) is very
weak. The results of the chi-square test and Spearman’s rank
correlation coefficient are presented in Table 3.
TABLE 3. THE RESULTS OF CHI-SQUARE TEST AND THE SPEARMAN’S
RANK CORRELATION COEFFICIENT
Self-efficacy
Spearman’s Rank
Correlation Coefficient
(rs Value)
Chi
-
Square
(P-Value)
Performance
0.11
0.61
Frequency of
pos sentiments 0.06 0.77
Intensity of pos
sentiments 0.06 0.75
As mentioned earlier the self-efficacy questions are
categorized into two themes of computer science and
cognition/social skills. In our data analysis, we investigate
which questions in each category are more related to the
performance. In other words which self-efficacy dimensions
have more potential to impact students’ performance. For this
purpose, we applied Principal Component Analysis (PCA)
method to identify the main dimensions that better represent
the self-efficacy with students’ performance as the target
value. PCA is an unsupervised statistical method for reducing
the feature space dimension to the most critical ones while
preserving as much ‘variability’ (i.e., statistical information)
as possible [34].
By creating a new feature (component) based on the linear
combination of initial features PCA reduces the number of
features in the dataset. The dimensionality reduction is one of
the main features of this method that makes it effective in
developing predictive models [36].
The steps to identify the best number of components for
maintaining the originality of the dataset are 1) identifying the
covariance matrix of the original 20 features in the self-
efficacy tool, 2) calculating the eigenvalue and eigenvector for
each feature, 3) sorting the eigenvalues in descending order
and 4) retaining the components that have eigenvalues higher
than 1.
Authorized licensed use limited to: Kennesaw State University. Downloaded on April 02,2024 at 17:21:10 UTC from IEEE Xplore. Restrictions apply.

The scatter plot of eigenvalues is presented in Fig. 6,
where the horizontal axis shows the features of the self-
efficacy tool, and the vertical axis shows the corresponding
eigenvalues. The result confirms we need only seven
components (highlighted in red color) to represent the self-
efficacy dataset.
Fig. 6. Scatter Plot of the Eigenvalues of the Principal Comp
To interpret each principal component in terms of the
original variables, we measure the magnitude of the
coefficients of the original 20 features. Larger absolute values
of coefficients indicate that the corresponding component
(i.e., the question in the self-efficacy tool) has more
importance in terms of the target value which is performance.
The self-efficacy questions related to the top seven principal
components are listed in Table 4. Since the maximum
coefficient values of the two components were related to one
question the total number of 6 questions are identified based
on the seven principal components.
TABLE 4. S-STEM QUESTIONS [27]
ID PRINCIPAL
COMPONENT
SELF-EFFICACY QUESTION
CS7 PC-4
PC-5
Computer science will be important to me
in my life’s work.
L&S2 PC-1 I am confident I can respect the
differences of my peers.
L&S4 PC-6 I am confident I can respect the
differences of my peers.
L&S9 PC-3 I am confident I can manage my time
wisely when working on my own.
L&S10 PC-2 When I have many assignments, I can
choose which ones need to be done first.
L&S11 PC-7 I am confident I can work well with
students from different backgrounds.
The result of our analysis shows that out of six questions
(each represented by a principal component), five of them
belong to the category of learning and social skills and only
one belongs to the computer science category. This confirms
that students' self-efficacy about their learning and social
skills plays a key role in their overall performance in the
course. This is an important finding since it cues instructors
and educators to pay particular attention to the students with
lower scores in these questions and provide effective
pedagogical interventions to improve their learning
experience.
The responses to the identified six self-efficacy questions
are plotted in the divergent stacked bar chart in Fig.7.
Fig. 7. Divergent Stacked Bar of the Self-Efficacy Questions
Fig.7 shows question L&S-9 has the most negative score
(i.e., disagree and strongly disagree) which asks if students
can manage their time when they work on their own, and
question L&S-11 has the highest positive score (i.e., strongly
agree) which shows students’ confidence in working with
other people with different backgrounds. This indicates that
students’ social skills and their interest in teamwork and
confidence in managing diversity positively impact how they
perform in the course.
V. CONCLUSION
In this study, we explore the impact of self-efficacy on
students’ performance in CS1 collaborative active learning
class. We also analyze the correlation between students’
positive emotions in low-stake team discussions and their self-
efficacy. To measure the level of students’ self-efficacy we
apply a standard self-report tool (S-STEM). This tool
measures students’ self-efficacy about computer science,
learning, and social skills. The sentiments are extracted from
students’ conversations as they work in low-stake teams in a
CS1 class and discuss the course content with their assigned
peers. By applying Natural Language Processing (NLP)
algorithm we conduct sentiment analysis and detect valence
and polarity of sentiments from the conversations. The output
of this algorithm the frequency and intensity of positive
sentiment scores in students’ speech during the semester.
By applying the chi-square test and Spearman’s rank
correlation coefficient method we test the hypothesis and
measure the intensity of association between parameters of
self-efficacy, performance, and emotion. The result of our data
analysis shows a statistically significant positive correlation
between self-efficacy and performance, which aligns with the
findings of the previous research. Data shows the correlation
of self-efficacy and students’ positive sentiments while
engaged in low-stake team discussions is statistically
significant but not as strong as the association between self-
efficacy and performance.
One of the main contributions of this study is to investigate
which sub-dimensions of self-efficacy most correlate with
performance. By applying the PCA method we reduce the
dimension of self-efficacy features to the most critical ones in
terms of performance as the target value. The dimensionality
reduction methods resulted in 7 principal components.
Accordingly, we identified the questions of self-efficacy that
most impact the performance score. Data shows that students’
self-efficacy in interpersonal skills and learning ability most
impact their performance. This finding is an important
Authorized licensed use limited to: Kennesaw State University. Downloaded on April 02,2024 at 17:21:10 UTC from IEEE Xplore. Restrictions apply.

support for collaborative active learning in which peer
learning is achieved through students’ communication.
The early result of our data analysis is promising to
conduct this method on a larger population of students and
derive more generic conclusions. We believe the finding of
this research can help educators to identify the students who
experience emotional obstacles or have low levels of self-
efficacy earlier in the semester and provide the required
assistance to them.
A. Futue work
In future work, we will apply this method to different
classes in computing and software engineering to analyze a
larger population of students. In the study design, we will have
more focus on team dynamics and attributes to pursue a
hypothesis that collaborative active learning can improve both
interpersonal skills and individual performance. We will
inspect the correlation of self-efficacy with performance and
emotion in teams consisting of only males or only females to
compare the result in each population. We will also consider
other sentiments and emotional states of students like joy,
happiness, anger, etc. in our analysis.
In this study, we employed the questions of the S-STEM
tool with a Likert Scale format. In the future study, we will
also include open-ended questions of this instrument and by
applying NLP methods extract information from students’
narratives for more in-depth analysis.
REFERENCES
[1] Arguedas, M., Daradoumis, T., & Xhafa, F. (2014, July). Towards an
emotion labeling model to detect emotions in educational discourse. In
2014 Eighth International Conference on Complex, Intelligent and
Software Intensive Systems (pp. 72-78). IEEE.
[2] Laguador, J. M. (2013). Developing students’ attitude leading towards
a life-changing career. Educational Research International, 1(3), 28-33.
[3] Bacay, T. E., Dotong, C. I., & Laguador, J. M. (2015). Attitude of
Marine Engineering Students on Some School-Related Factors and
their Academic Performance in Electro Technology 1 and 2. Studies in
Social Sciences and Humanities, 2(4), 239-249.
[4] Cohn, E., Cohn, S., Balch, D. C., & Bradley Jr, J. (2004). The relation
between student attitudes toward graphs and performance in
economics. The American Economist, 48(2), 41-52.
[5] Dehbozorgi, N., Maher, M. L., & Dorodchi, M. (2020, October).
Sentiment Analysis on Conversations in Collaborative Active Learning
as an Early Predictor of Performance. In 2020 IEEE Frontiers in
Education Conference (FIE) (pp. 1-9). IEEE.
[6] Munezero, M., Montero, C. S., Mozgovoy, M., & Sutinen, E. (2013,
November). Exploiting sentiment analysis to track emotions in
students' learning diaries. In Proceedings of the 13th Koli Calling
International Conference on Computing Education Research (pp. 145-
152). ACM.
[7] Wilson, K., & Narayan, A. (2016). Relationships among individual
task self-efficacy, self-regulated learning strategy use and academic
performance in a computer-supported collaborative learning
environment. Educational Psychology, 36(2), 236-253.
[8] Lishinski, A., Yadav, A., Good, J., & Enbody, R. (2016, August).
Learning to program: Gender differences and interactive effects of
students' motivation, goals, and self-efficacy on performance. In
Proceedings of the 2016 ACM Conference on International Computing
Education Research(pp. 211-220). ACM.
[9] Hórreo, V. S., & Carro, R. M. (2007, September). Studying the impact
of personality and group formation on learner performance. In
International Conference on Collaboration and Technology (pp. 287-
294). Springer, Berlin, Heidelberg.
[10] Ballen, C. J., Wieman, C., Salehi, S., Searle, J. B., & Zamudio, K. R.
(2017). Enhancing diversity in undergraduate science: Self-efficacy
drives performance gains with active learning. CBE—Life Sciences
Education, 16(4), ar56.
[11] Maras, P., & Kutnick, P. (1999). Emotional and behavioural difficulties
in schools: Consideration of relationships between theory and practice.
Social Psychology of Education, 3(3), 135-153.
[12] Aivaloglou, E., & Hermans, F. (2019, February). Early programming
education and career orientation: the effects of gender, self-efficacy,
motivation and stereotypes. In Proceedings of the 50th ACM Technical
Symposium on Computer Science Education (pp. 679-685).
[13] Vennila Ramalingam, Deborah LaBelle, and Susan Wiedenbeck. 2004.
Self-efficacy and Mental Models in Learning to Program. In
Proceedings of the 9th Annual SIGCSE Conference on Innovation and
Technology in Computer Science Education. ACM, 171–175.
[14] Felienne Hermans and Efthimia Aivaloglou. 2017. Teaching Software
Engineering Principles to K-12 Students: A MOOC on Scratch. In
Proceedings of the 39th International Conference on Software
Engineering: Software Engineering and Education Track. IEEE Press,
Piscataway, NJ, USA, 13–22.
[15] Kathryn M. Rich, Carla Strickland, T. Andrew Binkowski, Cheryl
Moran, and Diana Franklin. 2017. K-8 Learning Trajectories Derived
from Research Literature: Sequence, Repetition, Conditionals. In
Proceedings of the 2017 ACM Conference on International Computing
Education Research (ICER ’17). ACM, New York, NY,
USA, 182–190.
[16] Linda Seiter and Brendan Foreman. 2013. Modeling the Learning
Progressions of Computational Thinking of Primary Grade Students.
In Proceedings of the Ninth Annual International ACM Conference on
International Computing Education Research. ACM, 59–66.
[17] Christine Alvarado, Gustavo Umbelino, and Mia Minnes. 2018. The
Persistent Effect of Pre-College Computing Experience on College CS
Course Grades. In Proceedings of the 49th ACM Technical Symposium
on Computer Science Education (SIGCSE ’18). ACM, New York, NY,
USA, 876–881.
[18] Chris Wilcox and Albert Lionelle. 2018. Quantifying the Benefits of
Prior Programming Experience in an Introductory Computer Science
Course. In Proceedings of the 49th ACM Technical Symposium on
Computer Science Education (SIGCSE ’18). ACM, New York, NY,
USA, 80–85.
[19] Alex Lishinski, Aman Yadav, Jon Good, and Richard Enbody. 2016.
Learning to Program: Gender Differences and Interactive Effects of
Students’ Motivation, Goals, and Self-Efficacy on Performance. In
Proceedings of the 2016 ACM Conference on International Computing
Education Research. ACM, 211–220.
[20] Mary Beth Rosson, John M. Carroll, and Hansa Sinha. 2011.
Orientation of Undergraduates Toward Careers in the Computer and
Information Sciences: Gender, Self-Efficacy and Social Support.
Trans. Comput. Educ. 11, 3, Article 14 (Oct. 2011), 23 pages.
[21] Shari L. Britner and Frank Pajares. 2006. Sources of science self-
efficacy beliefs of middle school students. Journal of Research in
Science Teaching 43, 5 (2006), 485–499.
[22] Robert W Lent and Gail Hackett. 1987. Career self-efficacy: Empirical
status and future directions. Journal of Vocational Behavior 30, 3
(1987), 347 – 382.
[23] Jung, D. I., & Sosik, J. J. (1998). Group potency and collective
efficacy: Predictive validity, level of analysis, and effects of
performance feedback on future group performance. Paper In M. T.
Brannick, E. Salas, & C. W. Prince (Eds.), Team performance
assessment and measurement: Theory, methods, and applications (pp.
19–43). Mahwah, NJ, USA: Lawrence
[24] Malinauskas, R. K. (2017). Enhancing of Self-Efficacy in Teacher
Education Students. European Journal of Contemporary
Education, 6(4), 732-738.
[25] Christopher Watson, Frederick W.B. Li, and Jamie L. Godwin. 2014.
No Tests Required: Comparing Traditional and Dynamic Predictors of
Authorized licensed use limited to: Kennesaw State University. Downloaded on April 02,2024 at 17:21:10 UTC from IEEE Xplore. Restrictions apply.

Programming Success. In Proceedings of the 45th ACM Technical
Symposium on Computer Science Education. ACM, 469–474.
[26] Marcus Credé and L. Alison Phillips. 2011. A meta-analytic review of
the Motivated Strategies for Learning Questionnaire. Learning and
Individual Differences 21, 4 (2011), 337 – 346.
[27] Unfried, A., Faber, M., Stanhope, D. S., & Wiebe, E. (2015). The
development and validation of a measure of student attitudes toward
science, technology, engineering, and math (S-STEM). Journal of
Psychoeducational Assessment, 33(7), 622-639.
[28] Schwarzer, R., Bäßler, J., Kwiatek, P., Schröder, K., & Zhang, J. X.
(1997). The assessment of optimistic self‐beliefs: comparison of the
German, Spanish, and Chinese versions of the general self‐efficacy
scale. Applied Psychology, 46(1), 69-88.
[29] Chen, I. S. (2017). Computer self-efficacy, learning performance, and
the mediating role of learning engagement. Computers in Human
Behavior, 72, 362-370.
[30] Norman, G. (2010). Likert scales, levels of measurement and the
“laws” of statistics. Advances in health sciences education, 15(5), 625-
632.
[31] Hauke, J., & Kossowski, T. (2011). Comparison of values of Pearson's
and Spearman's correlation coefficients on the same sets of data.
Quaestiones geographicae, 30(2), 87-93.
[32] Mukaka, M. M. (2012). A guide to appropriate use of correlation
coefficient in medical research. Malawi medical journal, 24(3), 69-71.
[33] Bartimote-Aufflick, K., Bridgeman, A., Walker, R., Sharma, M., &
Smith, L. (2016). The study, evaluation, and improvement of university
student self-efficacy. Studies in Higher Education, 41(11), 1918-1942.
[34] Hess, A. S., & Hess, J. R. (2018). Principal component analysis.
Transfusion, 58(7), 1580.
[35] https://www.fi.ncsu.edu/pages/about-the-student-attitudes-toward-
stem-survey-s-stem/. Last accessed April 2021.
[36] Dehbozorgi, N. (2020). Sentiment Analysis on Verbal Data from Team
Discussions as an Indicator of Individual Performance (Doctoral
dissertation, The University of North Carolina at Charlotte).
Authorized licensed use limited to: Kennesaw State University. Downloaded on April 02,2024 at 17:21:10 UTC from IEEE Xplore. Restrictions apply.