Fig 11 - uploaded by Nasrin Dehbozorgi
Content may be subject to copyright.
Source publication
This full research paper studies affective states in students' verbal conversations in an introductory Computer Science class (CS1) as they work in teams and discuss course content. Research on the cognitive process suggests that social constructs are an essential part of the learning process [1]. This highlights the importance of teamwork in engin...
Citations
... Historically students' performances have been evaluated based on their knowledge of the course independent of their affective states. Research suggests that emotions play a critical role in the mental functions of the individual and can help in determining a person's behavior or performance in a given context [7]. Affective computing as a multidisciplinary field integrates different aspects of computer science, cognitive science, psychology, and artificial intelligence to develop systems that capture emotions via body language, facial recognition, speech recognition, and other behavioral patterns. ...
... Towards the application of SER in analysis of students performance in another study [7] the authors investigate the relationship between students' positive sentiments expressed during teamwork and their individual performance in the course. This study introduces the use of verbal conversations as a means to measure affective states by focusing on low-stake teams, which have less emphasis in performance evaluation compared to capstone teams. ...
... However, although the use of positive, negative, or neutral scale has been reported to be the most frequent association within threelevel polarity scales (Chiarello et al., 2020), a high dispersion in the measurements emerges. For example, by measuring extremely positive, positive, and non-positive sentiments (Zhang et al., 2012) or by using normalized compound between − 1 (extreme negative) and 1 (extreme positive) in Camacho and Goel (2018), Dehbozorgi et al. (2020), andOkoye et al. (2020). ...
... Hence, although only four studies present frontend solution, many other studies analyzed in this research acknowledge that the next steps in SA should go towards the development of more easy-to-use visual analytics tools for teachers and students, both at individual and at a social and collaborative levels, that would help instructors providing effective feedback interventions (Dehbozorgi et al., 2020;Le et al., 2018). This development would benefit from integrating usability tests for teachers to maximize SA tools for personalizing the information displayed by the user, such as dynamic filters (by students' group, by each student, by academic year…), dynamic grouping (so teachers can group their own charts and add notes) and dynamic data labels (so teachers can more easily read the data). ...
... Complementary, in four studies SA has been used to predict learning performance and possible withdrawals or dropouts early detection. Dehbozorgi et al. (2020) established a relationship between students' performance and a positive emotional climate. However, no correlation between students' negative sentiments and individual performance was measured, suggesting a need to develop more sophisticated predictive models. ...
Sentiment Analysis (SA), a technique based on applying artificial intelligence to analyze textual data in natural language, can help to characterize interactions between students and teachers and improve learning through timely, personalized feedback, but its use in education is still scarce. This systematic literature review explores how SA has been applied for learning assessment in online and hybrid learning contexts in higher education. Findings from this review show that there is a growing field of research on SA, although most of the papers are written from a technical perspective and published in journals related to digital technologies. Even though there are solutions involving different SA techniques that can help predicting learning performance, enhancing feedback and giving teachers visual tools, its educational applications and usability are still limited. The analysis evidence that the inclusion of variables that can affect participants’ different sentiment expression, such as gender or cultural context, remains understudied and should need to be considered in future developments.
... In another study [15], the researchers explore the application of SER in analyzing students' performance by investigating the relationship between students' positive sentiments expressed during teamwork and their individual performance in the course. This study focuses on low-stake teams, which have received less attention in performance evaluation compared to capstone teams, and proposes the use of verbal conversations as a means to measure affective states. ...
italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Contribution:
An AI model for speech emotion recognition (SER) in the educational domain to analyze the correlation between students’ emotions, discussed topics in teams, and academic performance.
Background:
Research suggests that positive emotions are associated with better academic performance. On the other hand, negative emotions have a detrimental impact on academic achievement. This highlights the importance of taking into account the emotional states of the students to promote a supportive learning environment and improve their motivation and engagement. This line of research allows the development of tools that allow educators to address students’ emotional needs and provide timely support and interventions.
Intended Outcome:
This work analyzes students’ conversations and their expressed emotions as they work on class activities in teams and investigates if their conversations are course-related or not by applying topic extraction to the conversations. Furthermore, a comprehensive analysis is conducted to identify the correlation between emotions expressed by students and the discussed topics with their performance in the course in terms of their grades.
Application Design:
The student’s performance is formatively evaluated, taking into account a combination of their scores in various components. The core of the developed model comprises a speech transcriber module, an emotion analysis module, and a topic extraction module. The outputs of all these modules are processed to identify the correlations.
Findings:
The findings show a strong positive correlation between the expressed emotions of “relief” and “satisfaction” with students’ grades and a strong negative correlation between “frustration” and grades. Data also shows a strong positive correlation between course-related topics discussed in teams and grades and a strong negative correlation between noncourse-related topics and grades.
... Traditional machine learning methods according to textual features such as Naïve Bayes classifier [35], NLTK vader [5], etc, have been proposed for ERC task [34]. As deep learning develops, many neural networks models are employed to classify the emotion at each utterance. ...
This paper deals with the utterance-level modalities missing problem with uncertain patterns on emotion recognition in conversation (ERC) task. Present models generally predict the speaker's emotions by its current utterance and context, which is degraded by modality missing considerably. Our work proposes a framework Missing-Modality Robust emotion Recognition (M2R2), which trains emotion recognition model with iterative data augmentation by learned common representation. Firstly, a network called Party Attentive Network (PANet) is designed to classify emotions, which tracks all the speakers' states and context. Attention mechanism between speaker with other participants and dialogue topic is used to decentralize dependence on multi-time and multi-party utterances instead of the possible incomplete one. Moreover, the Common Representation Learning (CRL) problem is defined for modality-missing problem. Data imputation methods improved by the adversarial strategy are used here to construct extra features to augment data. Extensive experiments and case studies validate the effectiveness of our methods over baselines for modality-missing emotion recognition on two different datasets.
... According to research, the poor level of an individual's success is triggered not only by the lack of cognitive capacity but also are impacted by the individual's attitude and lack of soft skills in both the educational and industrial realms [3]. As a result, it is important to consider different dimensions of the attitude like affect, behavior, and cognition (i.e., ABC of attitude [4]) into the curriculum to improve the students' learning experience and prepare them for the fourth industrial revolution [5]. Sentiment analysis is one way to capture emotional states and has been largely applied in the commercial domain, however during the past decade it drew the attention of researchers in the field of engineering education [6]. ...
... Different methods are proposed by educators to analyze students' emotions and sentiments in the academic setting [7]. The most common approaches to extract emotions are either text analysis on students' asynchronous discussions, and reflective writings [1] or speech analysis on their conversation about the course topics [5,24]. Some of the studies take a further step in analyzing emotions by conducting aspect-based emotion analysis that deals with emotions linked to each important aspect of the topic [8]. ...
... In the previous study [5], we conducted polarity-based sentiment analysis on collaborative verbal discussions in CS1 class to determine the relationship between positive sentiments and students' performance. Our analysis showed a strong positive correlation between students' positive sentiments in teams and their performance. ...
This full research paper focuses on a natural language processing (NLP) driven approach to extract emotions from the speech in collaborative learning environments and analyze how they correlate with the learner's performance. Social competency is one of the base competencies that have been the target of many educational researchers in engineering and computing education during the past several years. Studies show that the low level of individual's performance is not just due to lack of intellectual or cognitive competencies but also lack of social skills impact performance in both educational and industrial domain. For this reason, Engineering Education is encapsulating social skills into the curriculum to prepare students for the fourth industrial revolution (i.e., Industry 4.0). Towards this goal in earlier work [1], we proposed a model to identify the correlation between the sentiments extracted from students' speech in teams and their performance. The results of polarity sentiment analysis showed a strong positive correlation between students' positive feelings in teams and their individual performance in the course. This study takes a further step and conducts multi-class emotion analysis on students' speech in teams. The process consists of two steps:1) extracting different classes of sentiment such as joy, anger, anxiety, etc., and identifying their correlation with students' performance using collaborative speech in an introductory programming course (CS1), 2) Aspect-Based Emotion Analysis (ABEA). The approach we adopt is the supervised machine learning method and rule-based models on speech datasets. After pre-processing the text, we identify multi classes of sentiments. Aspect extraction is accomplished through the Part of Speech (POS) tagging, and patterns are extracted from the identified aspects. Finally, we use the combination of emotion classes and aspect patterns as feature vectors to train the K-Nearest Neighbor (KNN) algorithm to predict students' performance.
... Emotion and affective states are important aspects of attitude. According to research, positive emotions like joy, happiness, and satisfaction about the given subject positively influence students' learning experience [5] while emotional obstacles such as anger, anxiety, etc. can hinder their cognition process [6]. ...
... In order to measure students' self-efficacy, we apply the Student Attitudes Toward STEM (S-STEM) tool. Students' emotions are extracted by Natural Language Processing (NLP) methods from their collaborative speech in low-stake teams [5] in which the emphasis is on building communication skills and learning from peers rather than a group grade for the outcome. The students' individual grade in the course is considered as the performance metric. ...
... The algorithm for text mining and sentiment analysis from speech corpora is presented in our previous work [5]. This algorithm took the transcribed speech tokens as input and classified the sentiments in three classes of positive, negative, and neutral emotion as well as the compound value which is a unique metric calculated based on the three classes of emotion. ...
This full research paper studies the correlation of self-efficacy in computer science as well as learning and social skills with students' academic performance and their emotions in collaborative learning environments. Self-efficacy is an essential part of social cognitive theory and provides the foundation for analyzing human thoughts, motivations, and actions. Studies show that students' successful performance and accomplishment are directly affected by the level of self-efficacy. Therefore, analyzing self-efficacy in engineering education is important since it can impact the learning process in academic settings as well as provide a metric to track for improvement. Social cognitive theories also emphasize that students' interaction with each other affects their learning process and how they perform in educational settings. In previous work [5], we analyzed students' conversations in low-stake teams in an introductory programming course (CS1) and observed a strong positive correlation between students' positive emotions while interacting with each other with their performance in the course. In this study, we focus on the correlation of self-efficacy with learner's emotion and performance. We measure students' self-efficacy with a standard instrument called "Student Attitudes Toward STEM (S-STEM) Survey". For this purpose, we asked the participants to self-report on a 5-point Likert-scaled survey including 20 questions. These 20 questions are grouped into 2 main categories of computer science and learning/social skills. Students' emotions were extracted from their speeches in teams by applying natural language processing (NLP) methods. The result of data analysis shows a statistically significant correlation between overall self-efficacy and performance in the course and positive emotions during the teamwork. We further investigate which category of self-efficacy questions most correlate with students' performance. The result shows self-efficacy in interpersonal skills and learning ability most impact students' performance.
... Student-centered pedagogical practices have been applied by multiple educators in engineering education. One of the known forms of student-centered learning is collaborative active learning in which students mostly work in the form of low-stake teams [1], [4] to learn from each other and develop their interpersonal skills [1]. Active learning in engineering education improves students' computational thinking and problem-solving skills as well as long-term knowledge retention by 70% [2]. ...
... Student-centered pedagogical practices have been applied by multiple educators in engineering education. One of the known forms of student-centered learning is collaborative active learning in which students mostly work in the form of low-stake teams [1], [4] to learn from each other and develop their interpersonal skills [1]. Active learning in engineering education improves students' computational thinking and problem-solving skills as well as long-term knowledge retention by 70% [2]. ...
... Evaluating individual student's performance in teams is a complex process, especially in low-stake teams in which the product of teamwork does not have a large contribution to the students' grade [3]. In such teams, the main goal of teamwork is peer learning and developing interpersonal skills [1] [4]. Practicing low-stake teams is common in collaborative active learning classes [3]. ...
... The compound score is the adjusted normalized value of the sum of valence scores of each word in the lexicon. The equation of compound value is presented in Equation (1) [46]: (1) where sum_val is the sum of the sentiment arguments passed to the score_valence() function in the VADER algorithm. ...
Affective states, a dimension of attitude, have a critical role in the learning process. In the educational setting, affective states are commonly captured by self-report tools or based on sentiment analysis on asynchronous textual chats, discussions, or students’ journals.
Drawbacks of such tools include: distracting the learning process, demanding time and commitment from students to provide answers, and lack of emotional self-awareness which reduces the reliability. Research suggests speech is one of the most reliable modalities to capture emotion and affective states in real-time since it captures sentiments directly. This research, which is an extension of the work originally presented in FIE conference’20 [1], analyses students’ emotions during teamwork and explores the correlation of emotional states with students’ overall performance. The novelty of this research is using speech as
the source of emotion mining in a learning context. We record students’ conversations as they work in low-stake teams in an introductory programming course (CS1) taught in active learning format and apply natural language processing algorithms on the speech transcription to extract different emotions from conversations. The result of our data
analysis shows a strong positive correlation between students’ positive emotions as they work in teams and their overall performance in the course. We conduct aspect-based sentiment analysis to explore the themes of the positive emotions and conclude that the student’s positive feelings were mostly centered around course-related topics. The result of this analysis contributes to future development of predictive models to identify low performing students based on the emotions they express in teams at earlier stages of the semester in order to provide timely feedback or pedagogical interventions to improve their learning experience.
