![Regression of class conversation parameters on time [days] (Study 2, individual subjects)](https://www.researchgate.net/profile/Kai-Cortina/publication/261656154/figure/tbl1/AS:392399340752906@1470566701459/Regression-of-class-conversation-parameters-on-time-days-Study-2-individual-subjects_Q320.jpg)
![Regression of class conversation parameters on time [days]; (Study 4, individual data)](https://www.researchgate.net/profile/Kai-Cortina/publication/261656154/figure/tbl2/AS:392399340752907@1470566701486/Regression-of-class-conversation-parameters-on-time-days-Study-4-individual-data_Q320.jpg)
Content uploaded by Kai S Cortina
Author content
All content in this area was uploaded by Kai S Cortina on Aug 28, 2014
Content may be subject to copyright.
A preview of the PDF is not available
Using the LENA in teacher training: Promoting student involvement through automated feedback
Abstract and Figures
As part of professional development training for mathematic teachers, we used a speech recognition recorder (Language ENvironment Analysis, LENA) to create an automated teacher feedback system to help teachers monitor and limit the time they talk and to increase students’ active participation in mathematics lessons. Teachers got feedback with a 12h turnaround which allowed them to see how much they and their students talked on a daily basis. In this study, we wanted to know whether a) the system indicated change in the talk pattern, b) whether more complex indicators can be developed that are useful for professional development. Based on a pilot study we were able to implement an automated feedback of quality discussion episodes which was effective in increasing the amount of teacher-student discussion.
Figures - uploaded by Kai S Cortina
Author content
All figure content in this area was uploaded by Kai S Cortina
Content may be subject to copyright.
... Advances in Artificial Intelligence (AI) and Natural Language Processing (NLP) have great potential for analyzing instructional discourse and providing substantive feedback to support teacher learning. Possible applications include identifying different types of classroom activities [13,43,44,52] and providing automated feedback on various teacher discourse moves, such as moves designed to (a) guide discussion and ensure students' participation [20,50], (b) ask authentic questions; namely, questions for which the answers are not presupposed by the teacher [1,22] or (c) restate and use student ideas [10]. Traditional approaches to automated analysis of classroom discourse typically employ supervised machine learning (ML) methods combined with manual feature engineering and human expert annotation of collected data according to evaluation rubrics. ...
... These systems are capable of providing personalized indications to student teachers through automatic analysis of teacher speech or combining feedback and reflection within scenario-based learning activities. In the study of Wang, Miller and Cortina (2013), a speech recognition recorder, called LENA (Language ENvironment Analysis), was used to create an automated teacher feedback system to help teachers monitoring and limiting the time they talk, in order to increase students' active participation in lessons. In Bardach et al (2021), an online scenario-based learning was used to prepare student teachers for the classroom readiness and self-efficacy, providing feedback on authentic learning experiences that teachers can solve in everyday practice. ...
This paper aims to illustrate an automated system developed to give formative and personalized feedback to teachers in training. It is an expert system (Paviotti, Rossi & Zarka, 2012) that uses concrete examples, cases and scenarios to guide the engaged learners (Leake, 1996). In this regard, this system is able to create questionnaires, deliver them, collect and analyze data, send feedback to the participants to provide information about their beliefs and behaviors about teaching and learning processes. Far from constituting an assessment of teaching practices, the automated feedback demonstrates its usefulness in identifying teachers’ mindframes at an early stage, so as to be able to implement more specific and personalized training. This allows its application to be extended to further training areas as well as constituting an effective approach for need analysis and a preparatory action for numerous training activities (guided discussion with experts, observation on practice, modeling, etc.).
... Advances in Artificial Intelligence (AI) and Natural Language Processing (NLP) have great potential for analyzing instructional discourse and providing substantive feedback to support teacher learning. Possible applications include identifying different types of classroom activities [13,43,44,52] and providing automated feedback on various teacher discourse moves, such as moves designed to (a) guide discussion and ensure students' participation [20,50], (b) ask authentic questions; namely, questions for which the answers are not presupposed by the teacher [1,22] or (c) restate and use student ideas [10]. Traditional approaches to automated analysis of classroom discourse typically employ supervised machine learning (ML) methods combined with manual feature engineering and human expert annotation of collected data according to evaluation rubrics. ...
Engaging students in argument from evidence is an essential goal of science education. This is a complex skill to develop; recent research in science education proposed the use of simulated classrooms to facilitate the practice of the skill. We use data from one such simulated environment to explore whether automated analysis of the transcripts of the teacher's interaction with the simulated students using Natural Language Processing techniques could yield an accurate evaluation of the teacher's performance. We are especially interested in explainable models that could also support formative feedback. The results are encouraging: Not only can the models score the transcript as well as humans can, but they can also provide justifications for the scores comparable to those provided by human raters.
... Though this system was originally designed for language acquisition studies in children, it has also proved useful in data collection with adult populations. [25][26][27] This tool is composed of a small audio recorder (digital language processor [DLP]) designed to be slipped into a clothing pocket and accompanying software that analyzes the recording for metrics such as total adult word count, number of conversational turns, and a representation of the auditory environment in terms of the proportions of meaningful speech, distant speech, noise, and TV/electronic noise. In the current study, this software was used to isolate small sections of naturalistic conversational discourse from multi-hour recordings in order for us to analyze it manually. ...
Rating scales are frequently used in research and clinical practice with people with aphasia (PWA) to characterize communication in the home environment. However, it remains unclear whether responses provided on rating scales accurately reflect the communication that occurs. We aim to evaluate the accuracy of PWA's self-perceptions of verbal language use as measured by a rating scale and determine whether this accuracy is different from that of non–brain-injured (NBI) participants. Four PWA and four NBI participants completed a rating scale estimating their amount of verbal language production as compared with their communication partner. Audio recordings from participants' home environments were analyzed for proportion of words and conversational turns contributed by the participant, which were compared with rating scale estimates. Perceptions of verbal language output among both PWA and NBI participants showed variable accuracy, with discrepancies between estimates and objective data across both groups. The reliability of rating scales in quantifying language output appears questionable, suggesting they may not accurately represent naturalistic language environments of PWA. Additional research with larger sample sizes is warranted to investigate whether this trend is consistent across a larger population of individuals with aphasia.
... For example, working from recordings of speech from real classroom environments, D'Mello's research team developed models to reliably detect discursive features such as instructional talk, authentic teacher questions, elaborated evaluation, and uptake (Jensen et al., 2020;Kelly et al., 2018). Furthermore, Wang et al. (2013) highlight the promise of providing teachers with automated feedback on their discourse practices. They found that teachers who received automated feedback regarding the amount of teacher talk relative to student talk in their mathematics lessons significantly increased the quantity of student talk, suggesting that even basic information about teachers' discursive patterns, displayed in a readily accessible format, can produce changes in the desired directions. ...
Talk moves can promote conversations that enable students' equitable participation in a rigorous learning environment. This study examined teachers' perceptions of the usability, utility, and accuracy of a web-based application that provides automated feedback on their use of talk moves. Pilot data from 21 teachers indicates mainly positive perceptions, with a trend of increasing talk moves despite variability in teachers’ use of the application. A case study of the most frequent user illustrates an ideal use case. Overall, these findings point to the promise of automated tools to support meaningful professional learning opportunities for teachers, leading to instructional change.
... For example, Kelly and colleagues (2018) used both automatic speech recognition and machine learning to detect teachers' use of authentic questions, an important dimension of classroom discourse (Mehan, 1979;Tharp & Gallimore, 1991). Relatedly, Wang et al. (2013) used an automated speech recognition tool to classify the interaction patterns between teachers and students and provide timely feedback to teachers that could help them monitor students' active participation in classroom discussion. While both studies demonstrate the potential of computational techniques in measuring teaching practices in some ways, they only focus on a single aspect of teaching, and neither of them corroborates the relationship between computer-generated measures and classroom observation scores and student outcomes (Kane & Staiger, 2012). ...
Valid and reliable measurements of teaching quality facilitate school-level decision-making and policies pertaining to teachers. Using nearly 1,000 word-to-word transcriptions of fourth- and fifth-grade English language arts classes, we apply novel text-as-data methods to develop automated measures of teaching to complement classroom observations traditionally done by human raters. This approach is free of rater bias and enables the detection of three instructional factors that are well aligned with commonly used observation protocols: classroom management, interactive instruction, and teacher-centered instruction. The teacher-centered instruction factor is a consistent negative predictor of value-added scores, even after controlling for teachers’ average classroom observation scores. The interactive instruction factor predicts positive value-added scores. Our results suggest that the text-as-data approach has the potential to enhance existing classroom observation systems through collecting far more data on teaching with a lower cost, higher speed, and the detection of multifaceted classroom practices.
Prior studies have shown the importance of classroom dialogue in academic performance, through which knowledge construction and social interaction among students take place. However, most of them were based on small scale or qualitative data, and few has explored the availability and potential of big data collected from online classrooms. To address this issue, this paper analyzes dialogues in live classrooms of a large online learning platform in China based on natural language processing techniques. The features of interactive types and emotional expression are extracted from classroom dialogues. We then develop neural network models based on these features to predict high- and low-academic performing students, and employ interpretable AI (artificial intelligence) techniques to determine the most important predictors in the prediction models. In both STEM (science, technology, engineering, mathematics) and non-STEM courses, it is found that high-performing students consistently exhibit more positive emotion, cognition and off-topic dialogues in all stages of the lesson than low-performing students. However, while the metacognitive dialogue illustrates its importance in non-STEM courses, this effect cannot be found in STEM courses. While high-performing students in non-STEM courses show negative emotion in the last stage of lessons, STEM students show positive emotion.
examine 2 . . . contributors to nonoptimal training: (1) the learner's own misreading of his or her progress and current state of knowledge during training, and (2) nonoptimal relationships between the conditions of training and the conditions that can be expected to prevail in the posttraining real-world environment / [explore memory and metamemory considerations in training] (PsycINFO Database Record (c) 2012 APA, all rights reserved)
This study illuminates claims that teachers' mathematical knowledge plays an important role in their teaching of this subject matter. In particular, we focus on teachers' mathematical knowledge for teaching (MKT), which includes both the mathematical knowledge that is common to individuals working in diverse professions and the mathematical knowledge that is specialized to teaching. We use a series of five case studies and associated quantitative data to detail how MKT is associated with the mathematical quality of instruction. Although there is a significant, strong, and positive association between levels of MKT and the mathematical quality of instruction, we also find that there are a number of important factors that mediate this relationship, either supporting or hindering teachers' use of knowledge in practice.
merica's students are tested regularly, and the results serve as justification for closing schools, firing principals, awarding merits, and focusing professional development and curriculum reform. How- ever, little attention is paid to measuring the opportunities to learn that teachers create in classrooms—the quantity and quality of classroom instruction and relationships be- tween teachers and students. Nearly all state certification standards for "highly qualified teachers" focus on degree status (1, 2). However, despite evidence that teachers' instructional practices and relationships with students account for a substantial portion of the "added value" derived from attending school (3-5), observations of classroom experiences for large samples of students and teachers are limited. The Study Here, we describe results from a study of ele- mentary school classroom experiences for more than 1000 American children recruited at birth from 10 U.S. sites and enrolled in more than 2500 classrooms in more than 1000 elementary schools and 400 school dis- tricts. Our investigation approximates an epi- demiological study of opportunities to learn in U.S. classrooms, although the sample was not nationally representative. Of teachers observed, 90% were credentialed by their state; all had a bachelor's degree and 44% had a master's degree. These teachers meet many states' standards for highly qualified elemen- tary school teachers (1). Observation of classrooms complements student test performance as a measure of quality in the educational system and can document opportunities to learn that are reflected in students' performances on tests. We observed first, third, and fifth grades. First, we coded the presence of 44 behav- ioral events for 80 one-minute intervals. In a given school day, intervals were clustered into eight 10-minute cycles of coding, in which 30-second periods of observation alternated with 30-second periods of recording. Next, we rated nine dimensions of the quality of the emo- tional and instruc- tional climate on the basis of 20 minutes of observation, again across eight cycles during the day. We used these codes and ratings to assess opportunities to learn (6, 7).
After twenty-three years of teaching, I stepped out of the classroom and into the world of education research. As part of a team of researchers comparing mathematics teaching and learning in the United States and China, I spent many hours watching videotaped mathematics lessons from fourth- and fifth-grade classrooms in both countries. It was fascinating. (To be honest, it was luxurious, since I was not also trying to grade spelling tests while I watched.) After I got past my initial reactions to the foreign setting—including bare walls, desks in rows, and over forty students per class—more substantive features of the differences between Chinese classrooms and what I was accustomed to seeing in U.S. classrooms began to capture my attention.
To investigate relationships between teaching and learning mathematics, the six second-grade classrooms in one school were observed regularly during the 12 weeks of instruction on place value and multidigit addition and subtraction. Two classrooms implemented an alternative to the more conventional textbook approach. The alternative approach emphasized constructing relationships between place value and computation strategies rather than practicing prescribed procedures. Students were assessed at the beginning and the end of the year on place value understanding, routine computation, and novel computation. Students in the alternative classrooms, compared with their more traditionally taught peers, received fewer problems and spent more time with each problem, were asked more questions requesting them to describe and explain alternative strategies, talked more using longer responses, and showed higher levels of performance or gained more by the end of the year on most types of items. The results suggest that relationships between teaching and learning are a function of the instructional environment; different relationships emerged in the alternative classrooms than those that have been reported for more traditional classrooms.
This study analyzed 1-4-year-old children's attention to television as a function of age, sex, and the presence or absence of a number of relatively simple auditory and visual characteristics of a TV program. Children's attention and attributes of the "Sesame Street" program were rated and stored in a computer so that continuous information about the TV program could be related to continuous information about attention. There was an increase with age in attention to the TV, and attention was elevated in the presence of some attributes and depressed in the presence of others. Many of the attribute effects interacted with age, but very few interacted with the sex of the child.
Investigated the relationship between specific classroom behaviors and critical thinking. Four indicants of student involvement were measured using a modified version of N. A. Flanders's (1970) interaction analysis system: student participation, peer-to-peer interaction, faculty questions, and faculty encouragement and use of student ideas. Critical thinking was assessed by use of the Watson-Glaser Critical Thinking Appraisal and A. Chickering's (1972) behavioral self-report index. 12 college classrooms ( N = 138), equally divided among disciplines, were studied using canonical correlation, analysis of variance, and univariate analyses. Student participation, encouragement, and peer-to-peer interaction consistently emerged as being significantly and positively related to critical thinking. The results, though more suggestive than definitive, have significant implications for future research methodology and for faculty development efforts. (43 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
