ArticlePDF Available

Using the LENA in teacher training: Promoting student involvement through automated feedback

Authors:

Abstract and Figures

As part of professional development training for mathematic teachers, we used a speech recognition recorder (Language ENvironment Analysis, LENA) to create an automated teacher feedback system to help teachers monitor and limit the time they talk and to increase students’ active participation in mathematics lessons. Teachers got feedback with a 12h turnaround which allowed them to see how much they and their students talked on a daily basis. In this study, we wanted to know whether a) the system indicated change in the talk pattern, b) whether more complex indicators can be developed that are useful for professional development. Based on a pilot study we were able to implement an automated feedback of quality discussion episodes which was effective in increasing the amount of teacher-student discussion.
Content may be subject to copyright.
A preview of the PDF is not available
... For example, Kelly and colleagues (2018) used both automatic speech recognition and machine learning to detect teachers' use of authentic questions, an important dimension of classroom discourse (Mehan, 1979;Tharp & Gallimore, 1991). Relatedly, Wang, Miller, and Cortina (2013) used an automated speech recognition tool to classify the interaction patterns between teachers and students and provide timely feedback to teachers that could help them monitor students' active participation in classroom discussion. While both studies demonstrate the potential of computational techniques in measuring teaching practices in some ways, they only focus on a single aspect of teaching, and neither of them corroborates the relationship between computer-generated measures and classroom observation scores and student outcomes (Kane & Staiger, 2012). ...
... But the use of such methods in natural classroom settings remains rare. One exception is Wang, Miller, and Cortina's (2013) study, which featured an automatic feedback system using a speech recognition recorder for teachers with respect to teacher and student talk, silence, overlap talk, and episodes of quality discussion. The authors find that when teachers received timely feedback, they significantly reduced their talking and discussion time significantly increased. ...
Preprint
Full-text available
Valid and reliable measurements of teaching quality facilitate school-level decision-making and policies pertaining to teachers, but conventional classroom observations are costly, prone to rater bias, and hard to implement at scale. Using nearly 1,000 word-to-word transcriptions of 4th-and 5th-grade English language arts classes, we apply novel text-as-data methods to develop automated, objective measures of teaching to complement classroom observations. This approach is free of rater bias and enables the detection of three instructional factors that are well aligned with commonly used observation protocols: classroom management, interactive instruction, and teacher-centered instruction. The teacher-centered instruction factor is a consistent negative predictor of value-added scores, even after controlling for teachers' average classroom observation scores. The interactive instruction factor predicts positive value-added scores.
... For example, working from recordings of speech from real classroom environments, D'Mello's research team developed models to reliably detect discursive features such as instructional talk, authentic teacher questions, elaborated evaluation, and uptake (Jensen et al., 2020;Kelly et al., 2018). Furthermore, Wang et al. (2013) highlight the promise of providing teachers with automated feedback on their discourse practices. They found that teachers who received automated feedback regarding the amount of teacher talk relative to student talk in their mathematics lessons significantly increased the quantity of student talk, suggesting that even basic information about teachers' discursive patterns, displayed in a readily accessible format, can produce changes in the desired directions. ...
Article
Talk moves can promote conversations that enable students' equitable participation in a rigorous learning environment. This study examined teachers' perceptions of the usability, utility, and accuracy of a web-based application that provides automated feedback on their use of talk moves. Pilot data from 21 teachers indicates mainly positive perceptions, with a trend of increasing talk moves despite variability in teachers’ use of the application. A case study of the most frequent user illustrates an ideal use case. Overall, these findings point to the promise of automated tools to support meaningful professional learning opportunities for teachers, leading to instructional change.
... For example, Kelly and colleagues (2018) used both automatic speech recognition and machine learning to detect teachers' use of authentic questions, an important dimension of classroom discourse (Mehan, 1979;Tharp & Gallimore, 1991). Relatedly, Wang et al. (2013) used an automated speech recognition tool to classify the interaction patterns between teachers and students and provide timely feedback to teachers that could help them monitor students' active participation in classroom discussion. While both studies demonstrate the potential of computational techniques in measuring teaching practices in some ways, they only focus on a single aspect of teaching, and neither of them corroborates the relationship between computer-generated measures and classroom observation scores and student outcomes (Kane & Staiger, 2012). ...
Article
Valid and reliable measurements of teaching quality facilitate school-level decision-making and policies pertaining to teachers. Using nearly 1,000 word-to-word transcriptions of fourth- and fifth-grade English language arts classes, we apply novel text-as-data methods to develop automated measures of teaching to complement classroom observations traditionally done by human raters. This approach is free of rater bias and enables the detection of three instructional factors that are well aligned with commonly used observation protocols: classroom management, interactive instruction, and teacher-centered instruction. The teacher-centered instruction factor is a consistent negative predictor of value-added scores, even after controlling for teachers’ average classroom observation scores. The interactive instruction factor predicts positive value-added scores. Our results suggest that the text-as-data approach has the potential to enhance existing classroom observation systems through collecting far more data on teaching with a lower cost, higher speed, and the detection of multifaceted classroom practices.
... More recently, there has been explorations on systems to automatically generate feedback to teachers based on their class session recordings. For example, Wang et al. [81] used a wearable recording system for teachers to record classroom audio and did automatic analysis on turn-taking dynamics in classrooms. Ramakrishnan et al. [65] used classroom video and audio with deep learning techniques to automatically model classroom climate. ...
... These systems provide diagnostic information about basic features of classroom interaction such as amount of teacher talk, student talk, whole-class discussion, and student-led group work. There is evidence, at least with math classrooms, that providing teachers with automated feedback on such basic features can enhance their discourse practices (Wang, Miller, & Cortina, 2013) and improve student achievement (Wayne, Garet, Wellington, & Chiang, 2018). ...
Article
Full-text available
In this article, I offer a 50-year update on James Britton's (1969) seminal essay "Talking to Learn." I use Britton’s essay as a vehicle to take stock of the latest theory and empirical evidence on the power of talk for nurturing students’ literate futures. First, I revisit Britton’s essay and ask, "What did we know then about talk and learning?" Second, I ask, "What do we know now?" I examine features of productive talk, empirical studies of the benefits of talk for learning, the role of teachers in supporting such talk, and the strategic use of talk for different educational purposes. Third, I ask, "What does the future hold for talk and learning?" Here I review emerging developments in automated analysis of classroom discourse, dialogic pedagogy with very young children, and the use of digital technologies to augment dialogic pedagogy. I conclude by considering what the recent work on dialogic pedagogy means for literacy learning and teaching. Journal of Reading Recovery, 20(1), 33-48.
Article
Rating scales are frequently used in research and clinical practice with people with aphasia (PWA) to characterize communication in the home environment. However, it remains unclear whether responses provided on rating scales accurately reflect the communication that occurs. We aim to evaluate the accuracy of PWA's self-perceptions of verbal language use as measured by a rating scale and determine whether this accuracy is different from that of non–brain-injured (NBI) participants. Four PWA and four NBI participants completed a rating scale estimating their amount of verbal language production as compared with their communication partner. Audio recordings from participants' home environments were analyzed for proportion of words and conversational turns contributed by the participant, which were compared with rating scale estimates. Perceptions of verbal language output among both PWA and NBI participants showed variable accuracy, with discrepancies between estimates and objective data across both groups. The reliability of rating scales in quantifying language output appears questionable, suggesting they may not accurately represent naturalistic language environments of PWA. Additional research with larger sample sizes is warranted to investigate whether this trend is consistent across a larger population of individuals with aphasia.
Conference Paper
We focus on data collection designs for the automated analysis of teacher-student interactions in live classrooms with the goal of identifying instructional activities (e.g., lecturing, discussion) and assessing the quality of dialogic instruction (e.g., analysis of questions). Our designs were motivated by multiple technical requirements and constraints. Most importantly, teachers could be individually micfied but their audio needed to be of excellent quality for automatic speech recognition (ASR) and spoken utterance segmentation. Individual students could not be micfied but classroom audio quality only needed to be sufficient to detect student spoken utterances. Visual information could only be recorded if students could not be identified. Design 1 used an omnidirectional laptop microphone to record both teacher and classroom audio and was quickly deemed unsuitable. In Designs 2 and 3, teachers wore a wireless Samson AirLine 77 vocal headset system, which is a unidirectional microphone with a cardioid pickup pattern. In Design 2, classroom audio was recorded with dual first- generation Microsoft Kinects placed at the front corners of the class. Design 3 used a Crown PZM-30D pressure zone microphone mounted on the blackboard to record classroom audio. Designs 2 and 3 were tested by recording audio in 38 live middle school classrooms from six U.S. schools while trained human coders simultaneously performed live coding of classroom discourse. Qualitative and quantitative analyses revealed that Design 3 was suitable for three of our core tasks: (1) ASR on teacher speech (word recognition rate of 66% and word overlap rate of 69% using Google Speech ASR engine); (2) teacher utterance segmentation (F-measure of 97%); and (3) student utterance segmentation (F-measure of 66%). Ideas to incorporate video and skeletal tracking with dual second-generation Kinects to produce Design 4 are discussed.
Chapter
Full-text available
examine 2 . . . contributors to nonoptimal training: (1) the learner's own misreading of his or her progress and current state of knowledge during training, and (2) nonoptimal relationships between the conditions of training and the conditions that can be expected to prevail in the posttraining real-world environment / [explore memory and metamemory considerations in training] (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
This study illuminates claims that teachers' mathematical knowledge plays an important role in their teaching of this subject matter. In particular, we focus on teachers' mathematical knowledge for teaching (MKT), which includes both the mathematical knowledge that is common to individuals working in diverse professions and the mathematical knowledge that is specialized to teaching. We use a series of five case studies and associated quantitative data to detail how MKT is associated with the mathematical quality of instruction. Although there is a significant, strong, and positive association between levels of MKT and the mathematical quality of instruction, we also find that there are a number of important factors that mediate this relationship, either supporting or hindering teachers' use of knowledge in practice.
Article
Full-text available
merica's students are tested regularly, and the results serve as justification for closing schools, firing principals, awarding merits, and focusing professional development and curriculum reform. How- ever, little attention is paid to measuring the opportunities to learn that teachers create in classrooms—the quantity and quality of classroom instruction and relationships be- tween teachers and students. Nearly all state certification standards for "highly qualified teachers" focus on degree status (1, 2). However, despite evidence that teachers' instructional practices and relationships with students account for a substantial portion of the "added value" derived from attending school (3-5), observations of classroom experiences for large samples of students and teachers are limited. The Study Here, we describe results from a study of ele- mentary school classroom experiences for more than 1000 American children recruited at birth from 10 U.S. sites and enrolled in more than 2500 classrooms in more than 1000 elementary schools and 400 school dis- tricts. Our investigation approximates an epi- demiological study of opportunities to learn in U.S. classrooms, although the sample was not nationally representative. Of teachers observed, 90% were credentialed by their state; all had a bachelor's degree and 44% had a master's degree. These teachers meet many states' standards for highly qualified elemen- tary school teachers (1). Observation of classrooms complements student test performance as a measure of quality in the educational system and can document opportunities to learn that are reflected in students' performances on tests. We observed first, third, and fifth grades. First, we coded the presence of 44 behav- ioral events for 80 one-minute intervals. In a given school day, intervals were clustered into eight 10-minute cycles of coding, in which 30-second periods of observation alternated with 30-second periods of recording. Next, we rated nine dimensions of the quality of the emo- tional and instruc- tional climate on the basis of 20 minutes of observation, again across eight cycles during the day. We used these codes and ratings to assess opportunities to learn (6, 7).
Article
After twenty-three years of teaching, I stepped out of the classroom and into the world of education research. As part of a team of researchers comparing mathematics teaching and learning in the United States and China, I spent many hours watching videotaped mathematics lessons from fourth- and fifth-grade classrooms in both countries. It was fascinating. (To be honest, it was luxurious, since I was not also trying to grade spelling tests while I watched.) After I got past my initial reactions to the foreign setting—including bare walls, desks in rows, and over forty students per class—more substantive features of the differences between Chinese classrooms and what I was accustomed to seeing in U.S. classrooms began to capture my attention.
Article
To investigate relationships between teaching and learning mathematics, the six second-grade classrooms in one school were observed regularly during the 12 weeks of instruction on place value and multidigit addition and subtraction. Two classrooms implemented an alternative to the more conventional textbook approach. The alternative approach emphasized constructing relationships between place value and computation strategies rather than practicing prescribed procedures. Students were assessed at the beginning and the end of the year on place value understanding, routine computation, and novel computation. Students in the alternative classrooms, compared with their more traditionally taught peers, received fewer problems and spent more time with each problem, were asked more questions requesting them to describe and explain alternative strategies, talked more using longer responses, and showed higher levels of performance or gained more by the end of the year on most types of items. The results suggest that relationships between teaching and learning are a function of the instructional environment; different relationships emerged in the alternative classrooms than those that have been reported for more traditional classrooms.
Article
This study analyzed 1-4-year-old children's attention to television as a function of age, sex, and the presence or absence of a number of relatively simple auditory and visual characteristics of a TV program. Children's attention and attributes of the "Sesame Street" program were rated and stored in a computer so that continuous information about the TV program could be related to continuous information about attention. There was an increase with age in attention to the TV, and attention was elevated in the presence of some attributes and depressed in the presence of others. Many of the attribute effects interacted with age, but very few interacted with the sex of the child.
Article
Investigated the relationship between specific classroom behaviors and critical thinking. Four indicants of student involvement were measured using a modified version of N. A. Flanders's (1970) interaction analysis system: student participation, peer-to-peer interaction, faculty questions, and faculty encouragement and use of student ideas. Critical thinking was assessed by use of the Watson-Glaser Critical Thinking Appraisal and A. Chickering's (1972) behavioral self-report index. 12 college classrooms ( N = 138), equally divided among disciplines, were studied using canonical correlation, analysis of variance, and univariate analyses. Student participation, encouragement, and peer-to-peer interaction consistently emerged as being significantly and positively related to critical thinking. The results, though more suggestive than definitive, have significant implications for future research methodology and for faculty development efforts. (43 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)