Fig 8 - uploaded by Nasrin Dehbozorgi
Content may be subject to copyright.
Source publication
This full research paper studies affective states in students' verbal conversations in an introductory Computer Science class (CS1) as they work in teams and discuss course content. Research on the cognitive process suggests that social constructs are an essential part of the learning process [1]. This highlights the importance of teamwork in engin...
Contexts in source publication
Context 1
... Fig. 8", " Fig. 9" and " Fig. 10" visualize linear regression analysis of positive sentiment and subjectivity vs performance. In these plots the horizontal axis specifies the positive compound value (intensity and frequency) and subjectivity while the vertical axis shows performance score of each ...
Context 2
... " Fig. 8" and " Fig. 9", we observe a homogeneous pattern between positive sentiment (intensity and frequency) and performance (i.e. higher performance scores have higher positive sentiments). On the other hand, in " Fig. 10", the regression plot of the subjectivity and performance does not show any consistency between these two metrics. For ...
Citations
... However, although the use of positive, negative, or neutral scale has been reported to be the most frequent association within threelevel polarity scales (Chiarello et al., 2020), a high dispersion in the measurements emerges. For example, by measuring extremely positive, positive, and non-positive sentiments (Zhang et al., 2012) or by using normalized compound between − 1 (extreme negative) and 1 (extreme positive) in Camacho and Goel (2018), Dehbozorgi et al. (2020), andOkoye et al. (2020). ...
... Hence, although only four studies present frontend solution, many other studies analyzed in this research acknowledge that the next steps in SA should go towards the development of more easy-to-use visual analytics tools for teachers and students, both at individual and at a social and collaborative levels, that would help instructors providing effective feedback interventions (Dehbozorgi et al., 2020;Le et al., 2018). This development would benefit from integrating usability tests for teachers to maximize SA tools for personalizing the information displayed by the user, such as dynamic filters (by students' group, by each student, by academic year…), dynamic grouping (so teachers can group their own charts and add notes) and dynamic data labels (so teachers can more easily read the data). ...
... Complementary, in four studies SA has been used to predict learning performance and possible withdrawals or dropouts early detection. Dehbozorgi et al. (2020) established a relationship between students' performance and a positive emotional climate. However, no correlation between students' negative sentiments and individual performance was measured, suggesting a need to develop more sophisticated predictive models. ...
Sentiment Analysis (SA), a technique based on applying artificial intelligence to analyze textual data in natural language, can help to characterize interactions between students and teachers and improve learning through timely, personalized feedback, but its use in education is still scarce. This systematic literature review explores how SA has been applied for learning assessment in online and hybrid learning contexts in higher education. Findings from this review show that there is a growing field of research on SA, although most of the papers are written from a technical perspective and published in journals related to digital technologies. Even though there are solutions involving different SA techniques that can help predicting learning performance, enhancing feedback and giving teachers visual tools, its educational applications and usability are still limited. The analysis evidence that the inclusion of variables that can affect participants’ different sentiment expression, such as gender or cultural context, remains understudied and should need to be considered in future developments.
... Traditional machine learning methods according to textual features such as Naïve Bayes classifier [35], NLTK vader [5], etc, have been proposed for ERC task [34]. As deep learning develops, many neural networks models are employed to classify the emotion at each utterance. ...
This paper deals with the utterance-level modalities missing problem with uncertain patterns on emotion recognition in conversation (ERC) task. Present models generally predict the speaker's emotions by its current utterance and context, which is degraded by modality missing considerably. Our work proposes a framework Missing-Modality Robust emotion Recognition (M2R2), which trains emotion recognition model with iterative data augmentation by learned common representation. Firstly, a network called Party Attentive Network (PANet) is designed to classify emotions, which tracks all the speakers' states and context. Attention mechanism between speaker with other participants and dialogue topic is used to decentralize dependence on multi-time and multi-party utterances instead of the possible incomplete one. Moreover, the Common Representation Learning (CRL) problem is defined for modality-missing problem. Data imputation methods improved by the adversarial strategy are used here to construct extra features to augment data. Extensive experiments and case studies validate the effectiveness of our methods over baselines for modality-missing emotion recognition on two different datasets.
... According to research, the poor level of an individual's success is triggered not only by the lack of cognitive capacity but also are impacted by the individual's attitude and lack of soft skills in both the educational and industrial realms [3]. As a result, it is important to consider different dimensions of the attitude like affect, behavior, and cognition (i.e., ABC of attitude [4]) into the curriculum to improve the students' learning experience and prepare them for the fourth industrial revolution [5]. Sentiment analysis is one way to capture emotional states and has been largely applied in the commercial domain, however during the past decade it drew the attention of researchers in the field of engineering education [6]. ...
... Different methods are proposed by educators to analyze students' emotions and sentiments in the academic setting [7]. The most common approaches to extract emotions are either text analysis on students' asynchronous discussions, and reflective writings [1] or speech analysis on their conversation about the course topics [5,24]. Some of the studies take a further step in analyzing emotions by conducting aspect-based emotion analysis that deals with emotions linked to each important aspect of the topic [8]. ...
... In the previous study [5], we conducted polarity-based sentiment analysis on collaborative verbal discussions in CS1 class to determine the relationship between positive sentiments and students' performance. Our analysis showed a strong positive correlation between students' positive sentiments in teams and their performance. ...
This full research paper focuses on a natural language processing (NLP) driven approach to extract emotions from the speech in collaborative learning environments and analyze how they correlate with the learner's performance. Social competency is one of the base competencies that have been the target of many educational researchers in engineering and computing education during the past several years. Studies show that the low level of individual's performance is not just due to lack of intellectual or cognitive competencies but also lack of social skills impact performance in both educational and industrial domain. For this reason, Engineering Education is encapsulating social skills into the curriculum to prepare students for the fourth industrial revolution (i.e., Industry 4.0). Towards this goal in earlier work [1], we proposed a model to identify the correlation between the sentiments extracted from students' speech in teams and their performance. The results of polarity sentiment analysis showed a strong positive correlation between students' positive feelings in teams and their individual performance in the course. This study takes a further step and conducts multi-class emotion analysis on students' speech in teams. The process consists of two steps:1) extracting different classes of sentiment such as joy, anger, anxiety, etc., and identifying their correlation with students' performance using collaborative speech in an introductory programming course (CS1), 2) Aspect-Based Emotion Analysis (ABEA). The approach we adopt is the supervised machine learning method and rule-based models on speech datasets. After pre-processing the text, we identify multi classes of sentiments. Aspect extraction is accomplished through the Part of Speech (POS) tagging, and patterns are extracted from the identified aspects. Finally, we use the combination of emotion classes and aspect patterns as feature vectors to train the K-Nearest Neighbor (KNN) algorithm to predict students' performance.
... Emotion and affective states are important aspects of attitude. According to research, positive emotions like joy, happiness, and satisfaction about the given subject positively influence students' learning experience [5] while emotional obstacles such as anger, anxiety, etc. can hinder their cognition process [6]. ...
... In order to measure students' self-efficacy, we apply the Student Attitudes Toward STEM (S-STEM) tool. Students' emotions are extracted by Natural Language Processing (NLP) methods from their collaborative speech in low-stake teams [5] in which the emphasis is on building communication skills and learning from peers rather than a group grade for the outcome. The students' individual grade in the course is considered as the performance metric. ...
... The algorithm for text mining and sentiment analysis from speech corpora is presented in our previous work [5]. This algorithm took the transcribed speech tokens as input and classified the sentiments in three classes of positive, negative, and neutral emotion as well as the compound value which is a unique metric calculated based on the three classes of emotion. ...
This full research paper studies the correlation of self-efficacy in computer science as well as learning and social skills with students' academic performance and their emotions in collaborative learning environments. Self-efficacy is an essential part of social cognitive theory and provides the foundation for analyzing human thoughts, motivations, and actions. Studies show that students' successful performance and accomplishment are directly affected by the level of self-efficacy. Therefore, analyzing self-efficacy in engineering education is important since it can impact the learning process in academic settings as well as provide a metric to track for improvement. Social cognitive theories also emphasize that students' interaction with each other affects their learning process and how they perform in educational settings. In previous work [5], we analyzed students' conversations in low-stake teams in an introductory programming course (CS1) and observed a strong positive correlation between students' positive emotions while interacting with each other with their performance in the course. In this study, we focus on the correlation of self-efficacy with learner's emotion and performance. We measure students' self-efficacy with a standard instrument called "Student Attitudes Toward STEM (S-STEM) Survey". For this purpose, we asked the participants to self-report on a 5-point Likert-scaled survey including 20 questions. These 20 questions are grouped into 2 main categories of computer science and learning/social skills. Students' emotions were extracted from their speeches in teams by applying natural language processing (NLP) methods. The result of data analysis shows a statistically significant correlation between overall self-efficacy and performance in the course and positive emotions during the teamwork. We further investigate which category of self-efficacy questions most correlate with students' performance. The result shows self-efficacy in interpersonal skills and learning ability most impact students' performance.
... Evaluating individual student's performance in teams is a complex process, especially in low-stake teams in which the product of teamwork does not have a large contribution to the students' grade [3]. In such teams, the main goal of teamwork is peer learning and developing interpersonal skills [1] [4]. Practicing low-stake teams is common in collaborative active learning classes [3]. ...
... The compound score is the adjusted normalized value of the sum of valence scores of each word in the lexicon. The equation of compound value is presented in Equation (1) [46]: (1) where sum_val is the sum of the sentiment arguments passed to the score_valence() function in the VADER algorithm. ...
Affective states, a dimension of attitude, have a critical role in the learning process. In the educational setting, affective states are commonly captured by self-report tools or based on sentiment analysis on asynchronous textual chats, discussions, or students’ journals.
Drawbacks of such tools include: distracting the learning process, demanding time and commitment from students to provide answers, and lack of emotional self-awareness which reduces the reliability. Research suggests speech is one of the most reliable modalities to capture emotion and affective states in real-time since it captures sentiments directly. This research, which is an extension of the work originally presented in FIE conference’20 [1], analyses students’ emotions during teamwork and explores the correlation of emotional states with students’ overall performance. The novelty of this research is using speech as
the source of emotion mining in a learning context. We record students’ conversations as they work in low-stake teams in an introductory programming course (CS1) taught in active learning format and apply natural language processing algorithms on the speech transcription to extract different emotions from conversations. The result of our data
analysis shows a strong positive correlation between students’ positive emotions as they work in teams and their overall performance in the course. We conduct aspect-based sentiment analysis to explore the themes of the positive emotions and conclude that the student’s positive feelings were mostly centered around course-related topics. The result of this analysis contributes to future development of predictive models to identify low performing students based on the emotions they express in teams at earlier stages of the semester in order to provide timely feedback or pedagogical interventions to improve their learning experience.
This paper deals with the utterance-level modalities missing problem with uncertain patterns on emotion recognition in conversation (ERC) task. Present models generally predict the speaker's emotions by its current utterance and context, which is degraded by modality missing considerably. Our work proposes a framework Missing-Modality Robust emotion Recognition (M2R2), which trains emotion recognition model with iterative data augmentation by learned common representation. Firstly, a network called Party Attentive Network (PANet) is designed to classify emotions, which tracks all the speakers' states and context. Attention mechanism between speaker with other participants and dialogue topic is used to decentralize dependence on multi-time and multi-party utterances instead of the possible incomplete one. Moreover, the Common Representation Learning (CRL) problem is defined for modality-missing problem. Data imputation methods improved by the adversarial strategy are used here to construct extra features to augment data. Extensive experiments and case studies validate the effectiveness of our methods over baselines for modality-missing emotion recognition on two different datasets.
A picture is worth a thousand words, and in design metric estimation, a word may be worth a thousand features. Pictures are awarded this worth because they encode a plethora of information. When evaluating designs, we aim to capture a range of information, including usefulness, uniqueness, and novelty of a design. The subjective nature of these concepts makes their evaluation difficult. Still, many attempts have been made and metrics developed to do so, because design evaluation is integral to the creation of novel solutions. The most common metrics used are the consensual assessment technique (CAT) and the Shah, Vargas-Hernandez, and Smith (SVS) method. While CAT is accurate and often regarded as the “gold standard,” it relies on using expert ratings, making CAT expensive and time-consuming. Comparatively, SVS is less resource-demanding, but often criticized as lacking sensitivity and accuracy. We utilize the complementary strengths of both methods through machine learning. This study investigates the potential of machine learning to predict expert creativity assessments from non-expert survey results. The SVS method results in a text-rich dataset about a design. We utilize these textual design representations and the deep semantic relationships that natural language encodes to predict more desirable design metrics, including CAT metrics. We demonstrate the ability of machine learning models to predict design metrics from the design itself and SVS survey information. We show that incorporating natural language processing improves prediction results across design metrics, and that clear distinctions in the predictability of certain metrics exist.
