No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Measure for Measure: The Relationship between Measures of Instructional Practice in Middle School English Language Arts and Teachers’ Value-Added Scores
Abstract
Even as research has begun to document that teachers matter, there is less certainty about what attributes of teachers make the most difference in raising student achievement. Numerous studies have estimated the relationship between teachers' characteristics, such as work experience and academic performance, and their value-added to student achievement; but, few have explored whether instructional practices predict student test score gains. In this study, we ask what classroom practices, if any, differentiate teachers with high impact on student achievement in middle school English Language Arts from those with lower impact. In so doing, the study also explores to what extent value-added measures signal differences in instructional quality. Even with the small sample used in our analysis, we find consistent evidence that high value-added teachers have a different profile of instructional practices than do low value-added teachers. Teachers in the fourth (top) quartile according to value-added scores score higher than second-quartile teachers on all 16 elements of instruction that we measured, and the differences are statistically significant for a subset of practices including explicit strategy instruction.
... Studies in the past have provided evidence that teacher's instructional quality strongly linked with student performance (Depaepe & Ko¨nig, 2018;Ko¨nig & Pflanzl, 2016;Voss et al., 2011). This quality can also be used as a basis for measuring teacher effectiveness in school (Grossman et al., 2013). Ko¨nig et al. (2021) concluded that teacher instructional competencies serve as a significant predictor of qualityoriented mathematical teaching and learning processes and contribute to students' mathematical progress. ...
... Instructional competencies are considered part of teachers' professional competence (Kunter et al., 2013). This quality can also be used to measure teacher effectiveness in school (Grossman et al., 2013). Stovner et al. (2021) concluded that procedural feedback from teachers in the classroom proved to be more effective for students' understanding and behavior. ...
... Instructional competencies are considered part of teachers' professional competence (Kunter et al., 2013). This quality can also be used to measure teacher effectiveness in school (Grossman et al., 2013). ...
Professional development (PD) programs are considered as essential for improving the teacher quality and performance in schools. Despite introducing several PD programs, the teachers’ quality in the public schools is a matter of concern for Pakistan. It has become a challenge for policymakers and school leaders to design PD programs which can be effective for all teachers. PD programs in Pakistan normally undertakes a standard approach without considering the strengths and weaknesses of different groups of teachers which may not address the specific needs of these groups. There is a need of structuring PD programs to make these learning opportunities more effective for teachers. Therefore, the current study evaluates the teacher competencies of different groups of teachers in Pakistan’s public schools. For that purpose, the study first identifies three key themes of teacher competencies required in 21st century from extant review of recent literature. Interview guide then is designed based on these key teacher competencies to solicit information from seven principals of public schools. The public schools were selected using the criteria of size of school and principal working experience in that school. Structured interviews been used as a medium for collecting data from the principals. All interviews were recorded and transcribed verbatim. The deductive content analysis approach was used in analyzing the data. The analytical tool used for the coding of themes was the MAXQDA software. The results highlighted certain differences in teacher competencies varying from gender to age. Female teachers and teachers who are younger in general are found to have more competencies compared to male teachers and senior teachers. The results of this study offer vital information for planning and tailoring TPD programs for teachers. These specially designed learning opportunities can be more effective for the professional development of teachers and can foster teachers’ competence.
... We based the teaching dimensions and their respective categories on international research on what matters for student learning in science (e.g. Fauth et al., 2019;Neumann et al., 2012;Treagust & Tsui, 2014) and existing observation manuals for quality teaching (Grossman et al., 2013;Horton et al., 2009;Ødegaard et al., 2014). The overarching pillars build on a broad understanding of quality as transformation: 'the kind of transformative processes students (hopefully) go through in the course of their education,' (Wittek & Kvernbekk, 2011, p. 674). ...
... To operationalise quality teaching through observable categories, we developed an observation manual by combining three strategies from previous video studies (Neumann et al., 2012;Praetorius & Charalambous, 2018). First, we identified common features in existing observation manuals such as the Protocol for Language Arts Teaching Observation PLATO (Grossman et al., 2013) (PLATO; Grossman et al., 2013), the Electronic Quality of Inquiry Protocol EQUIP (Marshall et al., 2010), and the observation manual of the Budding Science and Literacy project (Ødegaard et al., 2014). Second, we selected relevant categories and adapted them to science teaching in line with the research literature on quality in science education (Fauth et al., 2019;Neumann et al., 2012;Treagust & Tsui, 2014). ...
... To operationalise quality teaching through observable categories, we developed an observation manual by combining three strategies from previous video studies (Neumann et al., 2012;Praetorius & Charalambous, 2018). First, we identified common features in existing observation manuals such as the Protocol for Language Arts Teaching Observation PLATO (Grossman et al., 2013) (PLATO; Grossman et al., 2013), the Electronic Quality of Inquiry Protocol EQUIP (Marshall et al., 2010), and the observation manual of the Budding Science and Literacy project (Ødegaard et al., 2014). Second, we selected relevant categories and adapted them to science teaching in line with the research literature on quality in science education (Fauth et al., 2019;Neumann et al., 2012;Treagust & Tsui, 2014). ...
A large body of research has studied the role and potential of inquiry to increase the quality of teaching in science education. While much of this existing research is based on international large-scale assessment studies, we still lack a clear understanding of the factors that influence the quality of inquiry-based science teaching in actual classroom practices. In this paper, we operationalise teaching quality through an observation manual, and we drew on this manual to systematically analyse video data of instructional practices in 20 Norwegian science classrooms at the primary and lower-secondary school level (73 observed lessons and about 450 students). We identified varying quality in the individual inquiry phases and differences between primary and lower-secondary schools. We observed that primary-school students collected and documented data more systematically than lower-secondary students and that consolidations were slightly more emphasised and of higher quality at the lower-secondary than at the primary level. Moreover, our findings indicate that inquiry-based teaching gave students more freedom to make their own choices and increased the quality of student participation in the classroom. Based on our findings, we discuss how teachers can improve the quality of inquiry-based instruction and empower students in the classroom.
... The existing literature on teacher effectiveness suggests that, after controlling for school and class variables, teachers are more important to student learning than any other factor (Rivkin, et al. 2005). However, knowledge about what works in teacher effectiveness is less well understood, as teachers significantly vary in their ability to improve students' performance (Hanushek and Rivkin, 2010;Grossman et al., 2013). Consequently, teacher effectiveness measures have been introduced to differentiate teacher performance better. ...
... Some descriptors are evaluated with a few instruments, while others are evaluated with several instruments. As we mentioned above, this is helpful, as there is a considerable agreement regarding the use of several instruments to evaluate teacher effectiveness due to single instruments hardly capturing all aspects of teacher quality (Grossman et al., 2013;Manzi et al., 2011). ...
... Given its importance, discourse quality represents a critical domain in structured classroom observation protocols, which are widely used by educational researchers to systematically evaluate teaching practices. These protocols such as CLASS (Classroom Assessment Scoring System [38]) and PLATO (Protocol for Language Arts Teaching Observations [20]) assess multiple aspects of teaching quality, including classroom management, social-emotional support, and instructional discourse. In this work, we adopt the observation protocol developed by the Global Teaching InSights (GTI) study [35], a largescale international classroom video study. ...
Classroom discourse is an essential vehicle through which teaching and learning take place. Assessing different characteristics of discursive practices and linking them to student learning achievement enhances the understanding of teaching quality. Traditional assessments rely on manual coding of classroom observation protocols, which is time-consuming and costly. Despite many studies utilizing AI techniques to analyze classroom discourse at the utterance level, investigations into the evaluation of discursive practices throughout an entire lesson segment remain limited. To address this gap, our study proposes a novel text-centered multimodal fusion architecture to assess the quality of three discourse components grounded in the Global Teaching InSights (GTI) observation protocol: Nature of Discourse, Questioning, and Explanations. First, we employ attention mechanisms to capture inter- and intra-modal interactions from transcript, audio, and video streams. Second, a multi-task learning approach is adopted to jointly predict the quality scores of the three components. Third, we formulate the task as an ordinal classification problem to account for rating level order. The effectiveness of these designed elements is demonstrated through an ablation study on the GTI Germany dataset containing 92 videotaped math lessons. Our results highlight the dominant role of text modality in approaching this task. Integrating acoustic features enhances the model's consistency with human ratings, achieving an overall Quadratic Weighted Kappa score of 0.384, comparable to human inter-rater reliability (0.326). Our study lays the groundwork for the future development of automated discourse quality assessment to support teacher professional development through timely feedback on multidimensional discourse practices.
... However, whether teacher constructs of teacher quality are in-born or can be developed through teacher development will have a significant impact in shaping evaluation system design (Gordon, et al., 2006). To respond to this, in the Understanding Teacher Quality (UTQ) study, teacher constructs of teacher quality were measured through an analysis of over 450 U.S. middle school teachers' practice through analysis of four standardised tools-CLASS (Hamre et al., 2012), FFT (Danielson, 2011), PLATO (Grossman et al., 2010), and MQI (Learning Mathematics for Teaching Project, 2006)-to assess a variety of dimensions of instruction, such as classroom management, motivation of students, and approaches in subjects. ...
This study examines effective instructional constructs grounded in Saudi Arabian English language teaching observations using Principal Component Analysis (PCA) that identifies four dimensions: adaptability, student engagement, instructional approaches, and instructional environment organisation. These dimensions together explain 62.32% of instructional practice variance and establish the multidimensionality of effective instruction. Dimension 1, contributing the most to variance (19.18%), encapsulates instructional structure and classroom management. These measures capture student engagement, the use of dynamic teaching strategies, and responsiveness to student diversity—crucial elements of pedagogical competence in English as Foreign Language (EFL) settings. Following models such as the Classroom Assessment Scoring System (CLASS) and the Framework for Teaching (FFT), the findings validate the critical need for robust teacher evaluation systems and related professional development. Based on systematic observation in the classroom and quantitative analysis, this study offers valuable lessons for teachers, school leaders, and policymakers who want to enhance instructional quality and student achievement.
... For example, the Classroom Assessment Scoring System (CLASS), originally developed at the OPEN ACCESS EDITED BY University of Virginia in the US (Pianta et al., 2008), has been widely used in countries across the globe as diverse as Australia (Thorpe et al., 2023), Chile (Leyva et al., 2015), China (Hu et al., 2016), Ecuador (Araujo et al., 2016), Finland (Virtanen et al., 2018), Germany (Bihler et al., 2018), the Netherlands (Slot et al., 2017), Norway (Westergård et al., 2019), Portugal (Cadima et al., 2010), and Singapore (Ng et al., 2021). Similarly, the Protocol for Language Arts Teaching Observation (PLATO) was developed at the University of Pennsylvania to assess English language arts instruction (Grossman et al., 2013) and later adopted by the Linking Instruction and Student Achievement (LISA) study to assess mathematics teaching in Nordic classrooms (Klette et al., 2017). ...
The current investigation aims to adapt and validate the Teacher Education and Development Study-Instruct observation instrument for assessing teaching quality in new contexts: Norwegian Grade 6 mathematics and science lessons. More specifically, the article examines content validity and reliability in the new contexts using a multi-methods approach, involving the Delphi technique and generalizability theory. Findings suggest that while the core components of the instrument are relevant in the new contexts, specific adaptations are necessary to capture teaching quality in a more nuanced and meaningful way. Based on the findings, specific adaptions are made to the instrument. Finally, recommendations for developing and using the instrument in the new contexts are provided. The current investigation underscores the importance of contextual sensitivity in the assessment of teaching quality.
... Ett generellt mönster är också att forskaren bidrar till att lyfta fram teoretiska begrepp som beskriver generella problem av undervisningspraktiken. Ett exempel på detta är när forskaren i ett av samtalen noterar det teoretiska begreppet "uptake" som bland annat Grossman (2013) använder för att beskriva och värdera hur lärare använder och fördjupar förståelsen av det elever säger eller skriver i undervisningen. Forskaren säger: I samtalet prövas att koppla ihop ett teoretiskt begrepp, "uptake", med lärarnas beprövade erfarenhet, där ett semantiskt rum för lärande skapas. ...
... 5. We consulted district leadership in EL and literacy who were familiar with the instructional practices used in the English as a Second Language, English Language Development, and English Language Arts classes in their districts. As well, we consulted survey research (Graham et al., 2014 national survey of writing instruction; Rowan & Correnti's, 2009 work on teachers' instructional logs) and research on assessing instructional quality in literacy (Grossman et al., 2013 on measuring language arts instruction) to ensure we were capturing activities reported in a wide range of contexts. As well, we consulted the IES Practice Guide on Adolescent Literacy (Kamil et al., 2008). ...
... Students in these lower secondary classrooms are generally aged 13-14. The database contains scores for the lessons according to the mathematics-adapted Protocol for Language Arts Teaching Observations (PLATO; see Grossman et al., 2013;Grossman, 2019). In PLATO, each 15-minute segment of a lesson receives a score for each element of teaching defined in the protocol. ...
Cognitive activation is a dimension of teaching quality which considers to what extent the teacher addresses the educational goal of student understanding, such as through successful implementation of demanding tasks. This study aimed to enrich empirical understandings of instructional formats and teacher-student interactions in cognitively activating lessons. Eight lessons were purposefully selected from a Nordic video database containing 125 lessons. The interactions in the lessons were analysed using reflexive thematic analysis and instructional formats using content analysis. Whole-class discussions and group work were the dominant formats. Four types of connection-making interactions were observed, connecting both within mathematics and to non-mathematical experience.
... A number of frameworks have been developed for investigating teaching quality and its impact on student learning (cf., Klette & Blikstad-Balas, 2018;Praetorius & Charalambous, 2018). These frameworks vary in the structure, the selection and naming of teaching dimensions, "the depth and breadth" (Senden et al., 2022, p. 151) of the teaching dimensions covered, and how subject matter is considered, ranging from generic frameworks designed to be applicable for all subjects (e.g., Classroom Assessment Scoring System [CLASS] by Pianta & Hamre, 2009) to those categorized as subject-specific as they were developed for a specific subject acknowledging its particularities (e.g., Mathematical Quality Instruction [MQI] by Hill et al., 2008, or the Protocol for Language Arts Teaching Observation [PLATO] by Grossman et al., 2013). ...
Recent publications emphasize the need to take greater account of differences in teaching quality between subjects. The empirical analysis of this topic requires a comparison of teaching quality in different subjects to distinguish generic aspects of teaching quality from subject-specific ones. In this paper, we compare teaching quality in mathematics and German lessons using observational data from primary schools in Switzerland ( N Math = 319; N German = 237). Data were collected using an observation instrument reflecting the teaching dimensions of the MAIN-TEACH model, which was developed based on a synthesis of established observation frameworks. The dimensions of classroom management, motivational-emotional support, selection and implementation of content, cognitive activation, support for consolidation, assessment and feedback, and adaptation were tested for subject-related measurement invariance. With a two-fold measurement invariance approach, differences between the subjects were investigated at both a global and an indicator level. When applying alpha accumulation correction, no significant subject-related differences in factor loadings or intercepts were found. The factorial structure of our data was basically identical for the two subjects. The comparison of latent factor means revealed no mean differences between the subjects. We discuss the implications for both the MAIN-TEACH model and research into subject-related differences of teaching in general.
... In addition to audio recording, a semi-structured observation checklist was employed. This was adapted from Grossman et al. (2013) Protocol for Language Arts Teaching Observations (PLATO 5.0) to scrutinise the teaching-learning practices in detail. The observation protocol consists of three instructional dimensions, such as quotation types (procedural, divergent, and convergent) and questioning strategies (probing, redirecting, repeating, and paraphrasing), listed with tick marks () indicating their level of evidence, categorised from 1 (little or not evident) to 4 (consistently evident). ...
ABSTRACTS
The objective of this study is to examine the questioning behaviours and strategies used by EFL instructors to enhance reflective learning in the communicative English language skills course at Hawassa University, Ethiopia. The descriptive qualitative case study was designed in line with the principles of the interpretivists’ research paradigm, which required qualitative data. To this effect, ten instructors were purposely selected to collect data through classroom observations and semi-structured interviews. Both data were audio-recorded. Then, the researchers transcribed the collected data verbatim. After transcribing, researchers thematically analysed the data. The findings showed that instructors frequently used convergent, divergent, and procedural questions, with probing, redirecting, and repeating as common strategies. Paraphrasing, prompting, and waiting time were less frequently used. Probing, redirecting, and waiting time were mainly used to enhance reflective learning. The study also revealed that many instructors were hesitant to use divergent questions during activities aimed at developing students’ language skills. Another limitation was the insufficient time for students to think and reflect on questions. This study suggests that instructors of the course should plan a variety of divergent questions before classes and allow adequate time for students’ reflection to encourage critical thinking and more idea generation.
... However, other than a few exceptions, these methods and related applications mostly focus on relatively low-inference instructional practices that rely on specific and concrete teaching moves (Rosenshine and Furst, 1971), such as the frequency of focusing (or open-ended) questions (Alic et al., 2022). In contrast, research is still scarce in exploring the feasibility of building automated measures on high-inference teaching practices, which require observers to draw conclusions and make interpretations based on many elements of teaching that are more aligned with existing classroom observation tools (Grossman et al., 2013). One exception is Wang and Demszky (2023), which uses zeroshot prompting to explore ChatGPT's ability in classifying classroom transcripts for a widely-used observation tool, Mathematical Quality Instruction (MQI) (Hill et al., 2008). ...
... Value-added is one of the most fundamental and precise indicators for assessing the effectiveness of an educational institution, categorizing and establishing trust in the credibility of its performance evaluation results [13]. The study by Grossman, Loeb, Cohen, and Wyckoff [39] indicated that utilizing the value-added approach in the field of education is highly effective in assessing teacher performance and enhancing the quality of education. ...
The current study aims to identify the degree of academic freedom practice and its relationship to the added scientific value and mobile technological forecasting from the perspective of faculty members in Saudi universities. To achieve this goal, a random sample of 23 faculty members from King Khalid University was selected from various specializations. A questionnaire was prepared to determine the extent of academic freedom exercised, the scale of scientific value added, and the scale of technological forecasting. The mean, standard deviation, and linear regression analyses were also used to analyze the research results. The results revealed that faculty members’ practice of academic freedom was average, while scientific value added and technological forecasting were high. The results also indicated the potential to predict the exercise of academic freedom through rapid access to information, financial resources, and technological forecasting.
... Besides, a structured observation checklist was used for further documentation and to validate the recording data. The scales were adopted from the protocol for language arts teaching observations (PLATO 5.0) by Ref. [51] and the design of teacher evaluation systems by Ref. [52]. Observable behaviors of EFL instructors' communication patterns are listed with tick marks (✓) indicating their level of evidence, categorized as: 1 = little or not evident, 2 = somewhat evident, 3 = evident, and 4 = very evident, as they occur in the EFL classroom. ...
Various studies have extensively examined classroom communication patterns in active language learning classrooms at higher education levels. However, research in the Ethiopian context is limited. While these studies contribute significantly to the field, they neglect to explore classroom communication patterns in reflective classroom practices. This study aimed to evaluate the communication patterns used by English as foreign language instructors in facilitating reflective learning within the Communicative English Language Skill Course at Hawassa University. Guided by constructivist theory, the research adopted a descriptive qualitative case study design within the interpretative paradigm. Ten certified Higher Diploma Program instructors teaching the course were purposefully selected for data collection through classroom observation and semi-structured interviews. Both datasets were audio-recorded, transcribed, and thematically analyzed. The findings revealed three communication patterns: instructorto-student, student-to-student, and student-to-instructor. The instructor-to-student pattern was the most commonly observed across all classes, characterized by instructors posing questions and students engaging in individual and collective reflection. However, the student-to-student pattern was rarely observed, particularly during group activities, and students often struggled to express their views due to time constraints. Additionally, there was limited communication between students and instructors, even when students had the opportunity to ask questions or give feedback. Despite instructors' efforts to establish rapport with students, reflective learning was not adequately facilitated through student-to-student and student-to-instructor communications. In conclusion, communication patterns in fostering reflective learning faced limitations, primarily due to insufficient opportunities for student reflection and inadequate time allocation for the reflection process, impacting its effectiveness.
... Promoting students' critical thinking skills through teachers' classroom instruction can improve students' learning outcomes (Barta et al., 2022;Lorencová et al., 2019), and there are different ways in which a teacher may do this. One particularly effective way is using open-ended questions, which have been found to be associated with student learning and participation in a growing number of studies from widely different contexts (Azigwe et al., 2016;Gill et al., 2016;Grossman et al., 2013;Hamre et al., 2014;Tyler et al., 2010), and which is a well-researched area (Simpson et al., 2014;Sattes, 2011, 2016). A seven-country study in sub-Saharan Africa found that less than one third of teachers use a mix of closed-and open-ended questions, with almost no questions that require students to apply what they learn in a different context (Bold et al., 2017). ...
Understanding and measuring effective teaching practices in low- and middle-income countries is a complex process that requires a contextualised knowledge of teaching quality, as well as adaptable instruments that can reliably capture teachers’ varied classroom behaviours. One approach developed with these purposes in mind is Teach Primary, a classroom observation framework designed and revised in 2021 by the World Bank. This framework captures the time teachers spend on learning and the quality of teaching practices which help to enhance pupils’ cognitive and socio-emotional skills, as well as aspects of the physical environment which may influence teaching quality. Using the Teach Primary framework, this article examines evidence regarding effective instructional practices in primary school classrooms, with highlighted examples from low- and middle-income countries. It also sheds light on the criticality of instrument contextualisation, given variations in how practices can be valued and implemented in different settings and how structural quality factors can influence teachers’ use of strategies. Challenges and limitations relating to the use of the Teach Primary framework are discussed, along with implications for teacher education and evaluation.
... Elf, 2021). The center makes use of a research design, developed by Klette, Blikstad-Ballas, and Roe (2017), which involves (1) non-probability sampling of classrooms and schools from different geographical areas, and with different socioeconomic backgrounds, (2) large scale video observations of lessons in the sampled classrooms, (3) administration of the Tripod student perception survey to students in the sampled classrooms (Ferguson, 2010), (4) collection of background data on the sampled teachers and schools, and (5) coding of the video observations by use of the Protocol for Language Arts Teaching Observations (PLATO) developed by Grossman et al. (2013). Though the PLATO manual was originally developed for observing and coding aspects of quality in English Language Arts teaching, the manual has subsequently been applied to teaching in Mathematics (Cohen, 2018) and ...
Classroom discussion constitutes a unique opportunity for student deliberation and is often believed to be a desirable attribute of Social Science teaching. Yet, despite its potential, large scale observational research on discussion in Nordic classrooms is scarce. To address this shortcoming, the present study investigated the prevalence and contexts of classroom discussion in a Nordic sample of 375 teaching segments (each lasting 15 min). The sample was gathered through video observations, and discussion was measured by an element from the Protocol for Language Arts Teaching Observations. Discussion was present in 39.2% of the investigated segments but was very unequally distributed across classes. While this study found classroom discussion to be most likely during small group work, in the middle of a 45-min lesson, and in schools with high achieving students, more research is needed to determine the sources of inequality in social science students’ opportunities for classroom deliberation.
... Two reliable and validated classroom observation instruments, the Mathematics-Scan (M-Scan; Berry et al., 2013;Walkowiak et al., 2014) and the Protocol for Language Arts Observations (PLATO; Corr, 2011;Grossman et al., 2013) guided our conceptualization of instructional activities. M-Scan and PLATO have been informed by many years of classroom-based research (see Section 2.1). ...
... While strategy instruction, modeling, and feedback hold great promise in scaffolding students' literacy development (Duke & Pearson, 2002;Graham, 2018;Graham & Perin, 2007;Hattie & Timperley, 2007;Pearson & Cervetti, 2015), previous research has indicated that time and experience are required for these practices to be applied successfully and that they are rarely enacted explicitly in language arts classrooms (Cohen, 2018;Grossman, Loeb, Cohen, & Wyckoff, 2013;Kane & Staiger, 2012;Klette, Blikstad-Balas, & Roe, 2017;Magnusson, Roe, & Blikstad-Balas, 2018;Tengberg et al., , 2022. Hence, teacher candidates need targeted support in understanding how to translate knowledge about these practices from university coursework into fieldwork (Alston & Danielson, 2021). ...
... Since the 1970s, increasing attention to the assessment of teachers' generic competencies has led to the development of a variety of observation instruments that are widely used to assess primary and secondary education, such as the Stallings Observe System (Stallings and Kaskowitz, 1974), the Framework for Teaching (Danielson, 1996), the International System for Teacher Observation and Feedback (Teddlie et al., 2006), the International Comparative Analysis of Learning and Teaching (Van de Grift, 2007), and the Classroom Assessment Scoring System (Pianta et al., 2008(Pianta et al., , 2011, among others. Although some recently developed observation instruments focus on more specific teaching competences, such as classroom conversation (Mercer, 2010), project-based learning (Stearns et al., 2012), and learning and instructional reform (Sawada et al., 2002), only a few observation instruments focus on teachers' strategies in specific subjects, such as reading in English (Gertsen et al., 2005), content and language integrated learning (De Graaff et al., 2007), English language arts (Grossman et al., 2010), and mathematics teaching (Matsumura et al., 2008;Hill et al., 2012;Schoenfeld, 2013). ...
This article focuses on the validation of two data collection instruments, one is the History Class Observation Tool (HCOT) and the other is focus groups with students, trainee teachers and tutors from secondary school classrooms. The main objective of the study is to obtain evidence of validity of the two instruments to be used in research on the design, validation, implementation, and evaluation of training units. All this in order to improve the teaching-learning process of historical thinking skills in high school students with the aim of forming critical citizens. The initial set of 32 observable behaviours was reviewed by 9 judges, who rated each on a scale of 0 (strongly disagree) to 3 (strongly agree) in terms of appropriateness, importance, clarity of wording, and observability. As can be seen from the results, both instruments according to the agreement indices suggest that the items, overall, can be considered suitable and important, and observable in the case of the observation instrument, as well as having clarity of wording appropriate to the objectives of both, with high confidence on the part of the experts. If we compare it with similar studies that we have discussed previously, we can see how this validation process has been quite rigorous and novel following some guidelines set previously in certain studies.
... These frameworks are generally grounded in the positivist paradigm relying on quantitative approaches. Examples of these include the Protocol for Language Arts Teaching Observation (PLATO; Grossman et al., 2013), the Classroom Assessment Scoring System (CLASS; Pianta et al., 2010), the Framework for Teaching (FfT; Danielson, 2007), the International System for Teacher Observation and Feedback (ISTOF; Muijs et al., 2018), the Global Teaching Insight (GTI Observation System (OECD, 2020)), the Teach Observation System (World Bank, 2022), and the International Comparative Analysis of Learning and Teaching instrument (ICALT; van de Grift, 2014). ...
Introduction
The purpose of this study was to examine measurement invariance in observer scoring of effective teaching behavior in three secondary education contexts–the Netherlands, England, and the United States. It also aimed to describe what effective teaching behavior looks like in secondary education across the three education contexts.
Methods
A uniform observation measure called International Comparative Analysis of Learning and Teaching (ICALT) was used to observe teachers.
Results
Results revealed that the hypothesized factor structure of effective teaching behavior was confirmed for the Dutch and English data, but not for the US data. Teachers in the Netherlands showed higher levels of more basic teaching behaviors, but lower levels of more complex teaching behaviors, compared to teachers in England.
Discussion
Implications of the findings are discussed.
... For example, consider the ways a teacher's confidence in their ability to teach math may shape student math classroom belonging. First, their beliefs may reflect the reality that teaching skills vary, that students in classes taught by more effective teachers experience greater academic success (Burgess et al., 2021;Grossman et al., 2013;Ronfeldt et al., 2015), and academic success feeds into greater feelings of belonging in the classroom-due in part to the confidence they gain from that success (Walton & Wilson, 2018;Yeager & Walton, 2011). Second, even net of actual skill, teachers who believe they are effective instructors may be more willing to try to reach all of their students than teachers who feel less effective. ...
Teachers set the tone for their classrooms, but what teacher beliefs shape students’ sense of belonging? We investigate how teachers’ mindsets—or their beliefs about learning and school—relate to adolescents’ individual and collective reports of classroom belonging. Our pre-registered analyses include a multilevel design of how 1,200 US middle school students (ages 11–13; 50% female; 49% low income; 40% White, 30% Latinx, 13% Black, 9% Asian) and their teachers responded to surveys on educational mindsets. We find teachers’ growth mindset and confidence in teaching positively relate to students’ math class belonging—explaining between 30 and 40% of belonging among classes. Our data suggest a teacher’s own sense of school belonging is unrelated to the belonging students feel in class, suggesting teachers’ broad feelings of belonging may not influence students’ specific classroom feelings of belonging as anticipated. These findings reinforce the notion that what teachers think and believe influence how students feel when in class.
... Thus, scholars constructed valid assessments targeted at different subjects. Examples of such classroom observation instruments included the Mathematical Quality of Instruction [17] and the Protocol for Language Arts Teaching Observation [18]. ...
This study revealed the current situation and developments in teacher evaluation in primary and secondary schools by reviewing 54 articles published in the recent decade (i.e., from January 2012 to October 2022). The coding scheme was developed based on the three components of effective teacher evaluation systems: “what”, “how”, and “who”. Specifically, we investigated the frameworks used for teacher evaluation, methods of evaluation, and participants in teacher evaluation. Based on our results, most studies evaluated teachers from the dimension of Instructional Support. Evaluation through video recording became popular due to technological advancement. Further, an increasing number of schools invited external experts to conduct teacher evaluations to ensure fairness. We also identified several crucial factors for teacher development: effective use of teaching resources and technology, high-quality feedback and communication, emotional support, classroom organization, and professional responsibilities. Due to COVID-19, many schools adopted distance learning, prompting the need to develop technological skills for teachers. Through the in-depth analysis of the current situation and development trends in the various dimensions of teacher evaluation in primary and secondary education, future research directions and issues were discussed and explored in this review.
... ding why organizational abilities are essential for teachers and studying some of the best talents, you may become more organized and enhance your classroom efficiency and time management. Consequently, recent research has also found several aspects of good classroom environments, such as teachers' organizing skills and relationships with students (Grossman et. al., 2013). Teachers serve as both a source of knowledge and a useful role model for their students in terms of strategies and behaviors. Therefore, it is their duty to impart declarative and procedural information (Runco & Nemiro, 1994). Being aware that the curriculum may present a planned opportunity for teachers must function as the primary li ...
This study focused on the acquired competencies of pre-service teachers in modular distance learning among 4 th-year BPED students currently enrolled in this semester S.Y. 2022-2023 in one of the universities of Eastern Visayas, Philippines. Thirteen (13) participants were selected using a purposive sampling technique based on the saturation of data. This study was qualitative in nature and employed a descriptive single-case study research design. The researcher conducted the study through video call via messenger for the interview. Data were collected using interviews and documents, and all data will be evaluated and interpreted using Thematic analysis. The study revealed the (1) Characteristics of pre-service teachers, (2) Opportunities in teaching, and (3) Acquired competencies of pre-service teachers. Thus, the pre-service teachers have acquired and utilized the skills and competencies in modular distance learning. Furthermore, the study concludes that the pre-service teachers are now ready for their in-service application.
... Klette et al. (2017) praesenterer internationale studier, der sammenfatter forskningen i fire basale dimensioner, der ifølge denne tradition er kritiske for høj kvalitet i undervisningen på tvaers af fag: a) klarhed i undervisningen (klare mål, tydelig instruktion og indholdsfokuseret undervisning); b) kognitiv aktivering (kvalitet i opgaver/laeringsressourcer, kognitiv udfordring, indholdsdaekning, adaptiv undervisning, begrebslig laering, flersprogethed), c) diskurstraek (kvalitet i laerer-elev-interaktion, elevens engagement) og d) støttende klima (styring af klasserummets procedurer, skabelse af et miljø praeget af respekt, respons og omsorg). Ud over denne generelle angelsaksiske tradition har de også arbejdet mere specifikt med kvalitet i undervisningen ved at transformere Pam Grossmans observationssystem PLATO (Protocol for Language Arts Teaching Observation) til en nordisk kontekst (Grossman et al., 2013). De anvender blandt andet PLATO som et faelles afsaet og et sprog om undervisning, der kan bruges til at fremme dialog om god undervisning frem for blot at levere svaret (Klette & Blikstad-Balas, 2018). ...
Hensigten med denne artikel er at præsentere en flerdimensionel model for perspektiver på kvalitet i undervisningen, der er udviklet i forbindelse med et større lodtrækningsforsøg, men som her for første gang undersøges i en kontekst, der gør det muligt at udfolde modellen og perspektivere dens teoretiske baggrund. Artiklen er af samme grund eksplorativ og teoriudviklende. Hensigten er a) at indkredse forskellige teoretiske og empiriske tilgange til kvalitet i undervisningen og b) at diskutere og underbygge den teoretiske model med basis i et fænomenologisk begreb om erfaret kvalitet i undervisningen. Det særlige fokus på en fænomenologisk fundering hænger sammen med forsøgets programteori. Antagelsen var, at der var behov for at styrke den oplevede kvalitet i undervisningen ved i langt højere grad at stilladsere og rammesætte skabende og undersøgende tilgange til fagenes indhold.
... This follows (Sinatra & Broughton, 2011), who stated that the characteristics of texts that may be associated with high interest include easy-to-understand, dense text, some depictions that seem alive, involve readers, cause various emotional reactions, and require prior knowledge. Wade et al. (Grossman et al., 2013)added other elements such as comprehension, novelty, and the value or importance of doing reading activities. ...
Introduction: Reading is one of the literacy skills needed in facing the challenges of the 21st century. Moreover, as a student, one who is an agent of change must have broad insight. The purpose of this study was to determine the reading interest of English Education study program students in the second semester of the academic year 2021/2020. Method: The method used in this research is descriptive quantitative. The number of research samples using random cluster sampling was 116 students. Result: From the results of statistical calculations, it was found that the mean value on the reading material indicator was more excellent at 2.68, meaning that reading material was the factor that most influenced students' reading interest. However, the reading material that students prefer is entertainment, such as story books and books containing romantic stories. In addition, reading has not become a student hobby. This is also indicated by the time they spend reading, which is only less than one hour. So, it is suggested that lecturers can continue to motivate students to foster students’ interest in reading, one of which is with a reading assignment system, then review and make a summary of the book that has been read. Conclusion: Reading material is the factor that most influences students' interest in reading. The types of books commonly read by students are entertainment books such as storybooks and books containing romantic stories. The time students read books every day is less than one hour. Reading has not become a hobby for students. The intensity of student visits to the library is rare, either during breaks or every day.
Hugsandi skólastofa (e. thinking classroom) er kennslunálgun sem snýst um að skapa rými þar sem nemendur vinna saman að verkefnum sem eru hönnuð til að styðja nemendur við að öðlast skilning á nýjum hugtökum og hugmyndum með samræðum. Í þessari rannsókn var fylgst með kennslu eins stærðfræðikennara í íslenskum framhaldsskóla. Markmið rannsóknarinnar var að veita kennaranum endurgjöf á gæði eigin kennslu þegar kennt er eftir hugmyndafræði hugsandi skólastofu og veita innsýn í sjálfsrýni hans um niðurstöðurnar. Gögnum var safnað með hljóðupptökum og áhorfi í þremur kennslustundum þar sem einum nemendahóp var fylgt eftir í eina kennsluviku. Gæði kennslunnar voru metin út frá greiningarrammanum PLATO fyrir yfirþættina faglegar kröfur (þ.e. samræður í skólastofunni og vitsmunaleg áskorun) og stigskiptan stuðning (þ.e. sýnikennsla, kennsla námsaðferða og endurgjöf). Helstu niðurstöður voru að í kennslustundunum voru faglegar kröfur almennt metnar á efri þrepum á meðan undirþættir stigskipts stuðnings voru metnir á bæði efri og neðri þrepum. Dæmi voru um sýnikennslu og kennslu námsaðferða á efsta þrepi, en endurgjöf var aldrei metin á efsta þrepi. Niðurstöðurnar sýna fram á að í íslensku samhengi er hægt að nota aðferðir hugsandi skólastofu til að búa til menningu þar sem nemendur vinna í sameiningu að krefjandi viðfangsefnum sem reyna á greinandi hugsun. Kennari hefur svigrúm til að biðja um rökstuðning og útskýringar. Það gefur nemendum tækifæri til að dýpka skilning sinn á verkefninu og lausnaferlinu. Samræður byggja á svörum nemenda og hafa þau tækifæri til að útskýra, segja frá og beita virkri hlustun. Hugsandi skólastofa virðist gefa tækifæri til að veita nemendum endurgjöf sem kennarinn hefði þó getað nýtt sér betur. Niðurstöður gefa einnig til kynna að hópumræða eftir vinnu og glímu nemenda við stærðfræðihugtök skapi aðstæður til sýnikennslu á efstu þrepum og sé heppilegri heldur en að hugtökin séu kynnt fyrir nemendum áður en þau vinna með þau
Teacher evaluation systems work to balance summative and formative goals. Instructional observations are an important part of such systems, allowing evaluation to directly impact instruction. Research shows that summative and formative goals can conflict with each other. This paper examines whether the summative usage of observation scores contributes to summative evaluation. We find that, after accounting for traditional principal ratings, observation scores provide little predictive validity for identifying high quality teachers. Given the reviewed negative impacts of summative uses of observation scores in teacher evaluation, teacher evaluation systems should reconsider the summative usage of observation scores.
The purpose of this study was to understand the nature of writing instruction across time and grade bands. We used quantitative and qualitative analyses of teacher interviews and video records of classroom instruction of English language arts writing instruction in 97 fourth- through eighth-grade classrooms in 2010 and 2018. Video records showed a decline in writing instruction across time and grade bands. Teacher lessons focused on the first five Common Core Writing Standards with little attention to the latter five. The lessons included aspects of a writing process approach that used instructional scaffolding, models, student practice, and teacher feedback. Lessons were less likely to include the use of authentic texts, text analysis, and student discussion. Teacher interviews pointed to curricular constraints, a view of writing as peripheral, and comfort with teaching writing as hindrances to quality writing instruction. These findings demonstrate a need for curricular and instructional resources to support teachers in offering consistently quality writing instruction.
Underpinning this consideration of writing instruction in Aotearoa New Zealand (NZ) is the premise that acts of teaching interact with the context in which they occur; they are shaped by the socio‐cultural milieu, philosophical and socio‐political traditions, curriculum and assessment systems, and the nature of individual classrooms. This perspective positions research regarding effective teaching and learning as requiring interpretation and, often, adaptation. Further, we have argued elsewhere that shared theories or understandings about constructs in writing instruction, applied within a context, can give rise to varied acts of instruction. Two constructs in writing instruction, key given features of the NZ context, are examined: developing independent, self‐regulating writers, and engaging in responsive, sustaining pedagogy. In NZ, shared theory of the importance of developing independent, self‐regulating writers is actioned in multiple pedagogical acts or approaches: teaching of strategies, largely through modelling; scaffolding goal setting; providing opportunities for decision making and choice; and enabling peer and self‐evaluation. Promoting self‐regulation is important given a policy of continuous intake, and traditions of non‐streamed classrooms and of teaching the individual. Shared understandings about responsiveness include knowing each individual student and building on, and sustaining, existing strengths. In teaching, writing this includes differentiating instruction often through the use of small‐group instruction, providing targeted, accessible feedback, and the use of culturally sustaining forms of instruction such as those involving trans‐languaging and storytelling. These understandings align with shared views of teaching as iterative inquiry and with official invitations to adapt curricula to fit local contexts.
Helping students engage with complex texts has been a longstanding challenge, though teachers have received little guidance about practices that help students in engaging with texts. This paper provides a range of empirical evidence about a tool designed to provide formative insight into text-focused teaching, which we used to reliably score more than 500 reading lessons in a large district. We describe the structure of the tool, its relationship to other measures of teaching, teacher and school attributes, and student outcomes. We then provide guidance to practitioners and researchers seeking to employ such a tool for teacher development.
The main objective of this study was to understand how teacher leadership and teacher quality impacted fourth grade student performance on the Louisiana Education Assessment Program (LEAP). The participants chosen were six schools in Louisiana who were labeled academically unacceptable for at least three years that taught fourth grade. A review of the data results provide an indication that, although standardized test scores of students are one piece of information for school leaders to use to make judgments about teacher effectiveness, such scores should be only a part of an overall comprehensive evaluation of the role of teacher leadership and teacher quality on student performance. The results varied by the characteristic of teacher quality and its impact on student performance.
We present the Teacher Talk Tool, which automatically analyzes classroom audio and provides formative feedback on key aspects of teachers’ classroom discourse (e.g., use of open-ended questions). The tool was designed to promote teacher learning by focusing attention and sense-making on their discourse. We conducted a feedback-response study where five English & Language Art teachers used the Teacher Talk Tool in eight classroom sessions. Teachers completed repeated-measure surveys and semi-structured interviews providing quantitative and qualitative evidence of feedback response. Results indicated that the majority of automated feedback was perceived to be accurate and prompted a high degree of reflection, focusing teachers’ attention on the measured talk constructs. This feedback also led teachers to engage in a process of sense-making, linking the measured talk features to classroom processes and contexts. However, evidence of feedback uptake was more limited. Overall, results contribute to the nascent literature on the efficacy of automated feedback on instructional practice.
The aim of this study was to assess the level of intellectual challenge in offered to students in lower secondary mathematics in Iceland as it appears in mathematical tasks and in the enactment of tasks in the classroom. Ten mathematics teachers participated in the study. A total of 34 mathematics lessons in 8th grade were recorded and analysed using an observation system (PLATO). Mathematical tasks in the lessons were analysed using the Task Analysis Guide. The results showed that a majority of mathematical tasks involved applying specific procedures to reach a solution. The teachers faced difficulties in upholding the level of challenge that the tasks offered. When students worked individually according to a list of numbered exercises it was common that teachers reduced the challenge that the tasks offered by asking only closed questions, computing for students or telling them the answer. The results also showed examples of solution of tasks where critical and creative thinking was required by the students and teachers encouraged collaboration and discussion. In those cases where intellectual challenge was at a high level, students worked together on the same tasks.
Denna rapport är genomförd i samarbete mellan Marcus Samuelsson och Joakim Samuelsson, som forskar om utveckling av skicklig undervisning, samt Bernt Friberg, Urban Hansson och Mats Rosenkvist som är experter på att utveckla evidensbaserad undervisningsskicklighet. Vår rapport bygger på en analys av en serie med 504 lektionsobservationer med fokus på undervisningsskicklighet som genomfördes med alla lärare på 30 grundskolor, 20 F-6 skolor och 10 7–9 skolor. Analysen visar på stor variation i undervisningens genomförande på samtliga skolor och utbredda brister i själva lärprocessen. Under 60 procent av de 504 lektionerna fanns inga lärandemål alls och lärandet synliggjordes sällan under lektionerna. Fokus i klassrummen var på att göra, inte att lära. Under cirka 10 procent av lektionerna hade eleverna förstått vad de skulle lära. Under cirka 45 procent av lektionerna använde läraren ett tydligt och konventionellt ledarskap med fokus på kontroll och sanktioner. Under endast 40 procent av lektionerna var lärarens ledarskap i klassrummet inriktat på att inspirera och motivera för lärande. Eleverna lämnades ofta ensamma att lyckas/misslyckas i arbetet under lektionerna och samlärande med andra elever användes i liten utsträckning. Tränade rutiner lyste med sin frånvaro i de flesta klassrum. Rapporten visar också att alla lärares undervisning blev signifikant bättre med metoden: lektionsobservationer, coachande samtal (”min undervisning”) och kollektiva reflektioner kring data om hur undervisningen genomfördes på skolan (”vår undervisning”). På 14 av de 30 skolorna observerades alla lärare, totalt 188, även en andra gång. Analysen av de observationerna visar att alla lärarnas undervisning då hade blivit signifikant bättre. Det innebär även att lärare som redan var skickliga hade blivit skickligare. Effekten mellan första och andra observationen varierade mellan medelstora och stora effekter för de 10 observerade dimensionerna. En annan sida av resultaten är lärarnas positiva reaktioner. Lärare som får möjlighet att strukturerat reflektera över undervisningsskicklighet i relation till sin undervisning på ett sätt som de känner är meningsfullt vill ha mer. Evidensbaserad undervisningsskicklighet som förmedlades till konkreta iakttagbara handlingar i klassrummet uppskattades eftersom lärarna kunde pröva dessa lärdomar redan nästa lektion. Rapporten visar att det med små resurser går att utveckla det som är framgångsrik evidensbaserad undervisningsskicklighet. Rapporten tydliggör för de som är ansvariga för undervisningen; politiker, rektorsutbildningar, lärarutbildningar, skolchefer, rektorer och lärare, hur brådskande och möjligt det är att utveckla undervisningen. 1,1 miljoner elever är i grundskolan här och nu. Studien kan fungera som inspiration till skolutveckling. Den kan påverka kultur och förhållningssätt inom den egna verksamheten, bidra till professionell dialog om undervisningens kvalitet samt leda till utveckling, mer lärande och högre kunskapsresultat på varje skola.
This study was conducted to collect validity evidence to support the use of an observation instrument to evaluate the performance of special education teachers (SETs) of students with significant disabilities (SWSD). In the study, a purposive sample of 49 SETs of SWSD, who were appropriately credentialed and experienced, evaluated the content of the Teacher Rating and Assessment Instrument for Teachers of Students with Significant Disabilities (TRAIT-SD) to determine each item’s importance for teaching SWSD. To do this, the participants were presented with the 37 items on the current version of the TRAIT-SD and asked to rate the importance of each using a 5-point Likert-type scale that ranged from very important to not important. They were also requested to provide narrative statements about the items and to recommend any relevant changes to the items, or recommend any new items they deemed to be missing from the current instrument. The results of the study showed that all participants, regardless of degree level, experience, or geographical location, rated all items between important and very important, resulting in all items being retained on the instrument. Additional comments resulted in a small number of items being reworded for clarity, and one new item being recommended for a subsequent version of the instrument.
Effective classroom management is critical to supporting student learning and success in school. However, little research has focused specifically on the impact of classroom management on student mathematics achievement, especially in the early elementary grades or in a small group (i.e., Tier 2) context. To address this need we investigated the relationship between classroom management and student mathematics achievement within the context of ROOTS, a validated Tier 2 kindergarten mathematics intervention taught by paraprofessionals. Results show that the quality of classroom management within the ROOTS intervention predicted gains in student mathematics achievement from pretest to posttest. These findings have critical implications for practitioners, including the importance of integrating effective classroom management techniques into Tier 2 mathematics intervention groups and supporting paraprofessionals' knowledge and use of effective classroom management.
Drawing on critical theories of labor and commodification, this qualitative embedded case study explores how students experience alienation and intimacy in the work of writing for an English language arts class. Analysis of fieldnotes from 30 observations, student writing products, and reflective interviews with focal students and the teacher illuminated the meaningful assemblages where conditions of intimacy permeated instruction. Two practices supported intimacy in working conditions: knowledge about writing built through a collective process of noticing, and open-ended work time characterized by “managed nonmanagement” (, p. 176), or calculated flexibility in rules and expectations. Findings illustrate how a literacy practice might contribute to students’ experience of alienation or intimacy (or both) while writing, depending on conditions of industrialization and commodification. Even as the teacher strove to deindustrialize work, commodification through grades and standardized assessments heightened alienation in the writing environment. The study provides an example of an educational context governed by an industrial system of assessment where local actors (the teacher and students) disrupted alienation by working in smaller scales and more closely with texts.
This report presents results from a survey of 64 elementary mathematics and reading language arts teacher educators providing feedback on a new type of short performance task. The performance tasks each present a brief teaching scenario and then require a short performance as if teaching actual students. Teacher educators participating in the study first reviewed six performance tasks, followed by a more in‐depth review of two of the tasks. After reviewing the tasks, teacher educators completed an online survey providing input on the value of the tasks and on potential uses to support teacher preparation. The survey responses were positive with the majority of teacher educators supporting a variety of different uses of the performance tasks to support teacher preparation. The report concludes by proposing a larger theory for how the performance tasks can be used as both formative assessment tools to support teacher learning and summative assessments to guide decisions about candidates' readiness for the classroom.
More than forty years ago, Dan Lortie famously lamented the lack of a common language with which to describe teaching and noted this scarcity as a key problem within studies of teaching and teacher professionalism. I argue that recent developments in video technology and supporting methodological designs have paved the way for a new generation of classroom studies. Today, large-scale classroom studies and targeted subject-specific studies have contributed to a renewed interest in classroom designs as lenses to shed light on how and why teaching and teaching matter. In this article, I summarize recent developments in classroom studies in terms of technologies, research design and analytical frameworks and discuss how these developments allow for integrative efforts and more programmatic research within studies of classroom teaching and learning, thus providing a platform for building a shared vocabulary for describing teaching. A key argument will be how video recordings nurture a new generation of generic and subject-specific classroom studies that enable us to systematically investigate key features of classroom teaching across subjects, grades, and learning environments.
This paper outlines the development of a comparative research framework in subject didactics and applies this in the process of analysing the transformations from academic disciplines across different school subjects. The theoretical framework builds on the concepts of ‘powerful knowledge’ and ‘transformation’ and ‘epistemic quality’ within which transformation processes from the classroom to the societal level are considered as ‘trajectories of powerful knowledge and epistemic quality”. The framework is used to analyse the findings from recent empirical studies across school subjects that have been reported on in publications arising from the Knowledge and Quality across School Subjects and Teacher Education (KOSS) network . The paper then focuses on analysing the transformations from disciplines across school subjects, given that the first boundary in defining powerful knowledge concerns knowledge that is specialized in both how it is produced and transmitted. To analyse this boundary the findings from the empirical studies are grouped into broad subject categories. These are then compared with the corresponding disciplines by using the widely cited Biglan classification scheme of academic disciplines in higher education. Finally, we consider the implications for curriculum planning and teacher education policy and reflect on the concept of subject-specific educational content knowledge (SSECK).
Focusing on systematic observation, one of the most potent methods of studying teaching quality, represents one of the numerous contributions of Daryl Siedentop to the profession. While he had a clear focus on issues of validity and reliability concerning systematic observation, over the past decades, attention to such issues appears to have eroded. By drawing on Siedentop’s contribution to systematic observation and the guidance he provided on gathering quality data, in this paper we discuss the lessons learned from his work regarding what needs to be observed, how, by whom, and under what conditions – all addressing important validity and reliability issues. We argue that closely attending to these issues is critical not only for reinvigorating a research field initiated by scholars such as Siedentop but also for restoring its key role in studying teaching quality.
Several recent, large-scale observations of naturally occurring classroom instruction have found consistent patterns in how teaching quality systematically varies across domains of instruction, patterns not found in student survey measures. Teaching quality is highest when examining aspects of classroom management; moderately high when examining aspects of student support; and low when focusing on the quality of the instructional support provided to students. This paper problematizes this interpretation, arguing that observed data patterns could result from dilemmas inherent in measuring teaching quality through observation. In problematizing these conclusions, we highlight the complexity of trying to measure instruction at scale, arguing for the need for caution and consideration of the complexity of measurement when interpreting scores from observation systems.
Classroom observations as lenses for studying and understanding features of teaching and learning processes hold a strong tradition in the educational sciences (see, for example, Brophy & Good, 1974; Callewaert & Nilsson, 1974; Flanders, 1970; Jackson, 1968, and Borgnakke, 1979; Klette, 1998, for Nordic studies). Currently, classroom observation design is a central part of educational researchers’ methodological repertoire for understanding instructional practices within and across different subjects and learning sites, thus providing a platform for comparative didactics (Caillot, 2007; Hudson & Meyer, 2011; Ligozat, 2011; Mercier et al., 2002; also see Ligozat, in this volume). The American ethnographer and classroom researcher Erickson (2006) identified the following traditional methods of observation studies targeted towards classroom learning: (i) discourse/interaction analyses, which are micro-analyses of language and communication; (ii) process/product approaches, which emphasize functional classroom interaction and activities; and (iii) teachers’ professional knowledge or pedagogic content knowledge, which is mainly on interaction and meaning-making of content. When subdividing this area of research into different school subjects; cultural, regional and linguistic traditions; and theoretical framings, we see a number of designs and frameworks used when aiming at analysing features of classroom teaching and learning. Today, classroom studies serve as the meeting ground for understanding teaching and learning processes from different approaches and perspectives. They diverse approaches capture different aspects of classroom learning such as interaction patterns, teachers’ use of scaffolding techniques, subject-specific aspects (i.e., how teachers represent a specific content or the quality of their explanations), and features of a supportive classroom climate. Despite voluminous research on classrooms, our knowledge so far is fragmented due to a multitude of single-case studies, across which it is hard to draw conclusions since the analytical framing and categories of the analyses used, as well as the contextual factors, differ substantially. In this chapter, I argue that efforts are needed to integrate the designs of classroom studies to serve as a platform for systematic investigation of key aspects of classroom teaching and learning. I refer to these integrated research efforts as ‘programmatic research’. As such the current chapter discusses methodologies for analysing teaching practices, and in a systematic way, arguing for how recent developments in observation designs—and especially improvements in observation instruments and development in video capture—have paved the way for a new area of comparative didactics.
This paper examines two kinds of student engagement: "procedural," which concerns classroom rules and regulations; and "substantive," which involves sustained commitment to the content and issues of academic study. It describes the manifestations of these two forms of engagement, explains how they relate differently to student outcomes, and offers some empirical propositions using data collected on literature instruction, collected during 1987-88 from 58 eighth-grade English classes (N=1,041 students). The results provide support for the following three hypotheses: (1) disengagement adversely affects achievement; (2) procedural engagement has an attenuated relationship to achievement because its observable indicators conflate procedural and substantive engagement; and (3) substantive engagement has a strong, positive effect on achievement. Features of substantively engaged instruction include authentic questions or questions that have no prespecified answers; uptake or the incorporation of previous answers into subsequent questions; and high-level teacher evaluation or teacher certification and incorporation of student responses in subsequent discussion. Each of these features is noteworthy because it involves reciprocal interaction and negotiation between students and teachers, which is said to be the hallmark of substantive engagement. (Author/TJH)
Offering perspectives focused on the meaningful goal of measuring and assessing teacher quality, Measurement Issues and Assessment for Teaching Quality brings together leading researchers and practitioners from the fields of education, economics and policy who contribute provocative, illuminating, and coherent articles that explore key issues surrounding this vital topic.
While teacher content knowledge is crucially important to the improvement of teaching and learning, attention to its development and study has been uneven. Historically, researchers have focused on many aspects of teaching, but more often than not scant attention has been given to how teachers need to understand the subjects they teach. Further, when researchers, educators and policy makers have turned attention to teacher subject matter knowledge the assumption has often been that advanced study in the subject is what matters. Debates have focused on how much preparation teachers need in the content strands rather than on what type of content they need to learn. In the mid-1980s, a major breakthrough initiated a new wave of interest in the conceptualization of teacher content knowledge. In his 1985 AERA presidential address, Lee Shulman identified a special domain of teacher knowledge, which he referred to as pedagogical content knowledge. He distinguished between content as it is studied and learned in disciplinary settings and the "special amalgam of content and pedagogy" needed for teaching the subject. These ideas had a major impact on the research community, immediately focusing attention on the foundational importance of content knowledge in teaching and on pedagogical content knowledge in particular. This paper provides a brief overview of research on content knowledge and pedagogical content knowledge, describes how we have approached the problem, and reports on our efforts to define the domain of mathematical knowledge for teaching and to refine its sub- domains.
Abstract In this study we attempted to illuminatewhy,measures of instruction sometimes fail to meet discrete tests of validity. We used a triangulation strategy—multiple methods, data sources, and researchers—toinvestigate teachers' and observers' reports on a daily language arts log. Data came from a pilot study of the log conducted in 8 urban public elementary schools. Statistical results increased our confidence in the log's ability to measure:a) instruction at grosser levels of detail, b) instructionalactivities that occurred more frequently, and c) word analysis instruction. Some qualitative evidence gave us greater confidence in the instrument-for example, when teachers differed from observers because they possessed background,knowledge,not available to observers. Other qualitative evidence illustrated dilemmas inherent inmeasuring instruction. Overall, we
This study investigates the nature of pedagogical expertise by comparing the planning, teaching, and postlesson reflections of three student teachers (two secondary and one elementary) with those of the cooperating teachers with whom they were placed. Participants were observed teaching mathematics for 1 week of instruction and were interviewed prior to and following each lesson. Differences in the thinking and actions of these experts and novices were analyzed by perceiving teaching both as a complex cognitive skill and as improvisational performance Novices showed more time-consuming, less efficient planning, encountered problems when attempts to be responsive to students led them away from scripted lesson plans, and reported more varied, less selective postlesson reflections than experts. These differences were accounted for by the assumptions that novices' cognitive schemata are less elaborate, interconnected, and accessible than experts' and that their pedagogical reasoning skills are less well developed. We offer several recommendations for student teaching based on this analysis.
In this article we examine methodological and conceptual issues that emerge when researchers measure the enacted curriculum in schools. After outlining key theoretical considerations that guide measurement of this construct and alternative strategies for collecting and analyzing data on it, we illustrate one approach to gathering and analyzing data on the enacted curriculum. Using log data on the reading/language arts instruction of more than 150 third-grade teachers in 53 high-poverty elementary schools participating in the Study of Instructional Improvement, we estimated several hierarchical linear models and found that the curricular content of literacy instruction (a) varied widely from day to day, (b) did not vary much among students in the same classroom, but (c) did vary greatly across classrooms, largely as the result of teachers' participation in 1 of the 3 instructional improvement interventions (Accelerated Schools, America's Choice, and Success for All) under study. The implications of these findings for future research on the enacted curriculum are discussed.
In this study we investigated the effects of teaching on students' reading achievement. More specifically, based on a framework of reading instruction maximizing students' cognitive engagement in literacy learning, curricular and teaching variables, including aspects of word-recognition and comprehension instruction, approaches to teaching such as telling versus coaching, and the enabling of students' active versus passive responding to literacy activities, were investigated to explain growth on reading comprehension, fluency, and writing measures over a school year in grade 1-5 classrooms. Participants included 88 teachers and 9 randomly selected students per classroom in 9 high-poverty schools across the United States that were engaged in a literacy instruction reform project. Teachers were observed 3 times across the school year during a reading lesson. Hierarchical linear modeling showed that a number of teaching variables explained substantial variation in student growth on several measures of reading achievement. Looking across all of the data, the most consistent finding was that teachers who emphasized higher-order thinking, either through the questions they asked or the tasks they assigned, promoted greater reading growth among the 9 target students in their classrooms. We examine the results of our work in relation to a framework of teacher effectiveness maximizing students' cognitive engagement in literacy learning.
This article describes the development of artifact collection and scoring procedures to characterize classroom practice in mathematics and science. A data collection tool called the "Scoop Notebook" was used to gather artifacts related to key features of classroom practice, such as teachers' use of instructional materials and strategies, classroom learning activities, and students' work. Pilot studies were conducted in a small number of middle school science and mathematics classrooms to provide initial information about the reliability, validity, and feasibility of artifact collections as measures of classroom practice. The pilot studies yielded positive results, indicating that the Scoop Notebook and associated scoring guide have promise for providing accurate representations of what teachers and students do in classrooms. Due to the small sample size, results are most appropriately used formatively, to help improve artifact collection procedures before they are tested on a larger scale.
merica's students are tested regularly, and the results serve as justification for closing schools, firing principals, awarding merits, and focusing professional development and curriculum reform. How- ever, little attention is paid to measuring the opportunities to learn that teachers create in classrooms—the quantity and quality of classroom instruction and relationships be- tween teachers and students. Nearly all state certification standards for "highly qualified teachers" focus on degree status (1, 2). However, despite evidence that teachers' instructional practices and relationships with students account for a substantial portion of the "added value" derived from attending school (3-5), observations of classroom experiences for large samples of students and teachers are limited. The Study Here, we describe results from a study of ele- mentary school classroom experiences for more than 1000 American children recruited at birth from 10 U.S. sites and enrolled in more than 2500 classrooms in more than 1000 elementary schools and 400 school dis- tricts. Our investigation approximates an epi- demiological study of opportunities to learn in U.S. classrooms, although the sample was not nationally representative. Of teachers observed, 90% were credentialed by their state; all had a bachelor's degree and 44% had a master's degree. These teachers meet many states' standards for highly qualified elemen- tary school teachers (1). Observation of classrooms complements student test performance as a measure of quality in the educational system and can document opportunities to learn that are reflected in students' performances on tests. We observed first, third, and fifth grades. First, we coded the presence of 44 behav- ioral events for 80 one-minute intervals. In a given school day, intervals were clustered into eight 10-minute cycles of coding, in which 30-second periods of observation alternated with 30-second periods of recording. Next, we rated nine dimensions of the quality of the emo- tional and instruc- tional climate on the basis of 20 minutes of observation, again across eight cycles during the day. We used these codes and ratings to assess opportunities to learn (6, 7).
State accountability systems and the federal No Child Left Behind Act have put additional demands on schools and teachers to improve teacher quality and improve student achievement. Many researchers (e.g., Cohen, 1996; Corcoran & Goertz, 1995; Floden, 1997; Newman, King, & Rigdon, 1997) have argued that such improvements will require a substantial increase in the instructional capacity of schools and teachers. One strategy for capacity building is to provide teachers with incentives to improve their performance, knowledge, or skills. The incentive strategy requires the design and implementation of alternative teacher compensation systems that depart from the single salary schedule (Odden, 2000; Odden & Kelley, 2002). Though slow to take hold, the incentive strategy is currently being pursued by several states (Peterson, 2006). Most of these new or proposed plans link pay to combinations of assessments of teacher performance, acquisition of new knowledge and skills, and student test score gains. Denver's widely followed Pro Comp plan also contains these components.
The Teacher Compensation Group of the Consortium for Policy Research in Education (CPRE) has been studying the design and effectiveness of such systems for nearly a decade. We initially focused on school-based performance award programs, in which each teacher in a school receives a bonus for meeting or exceeding schoolwide student achievement goals (Heneman, 1998; Heneman & Milanowski, 1999; Kelley, Heneman, & Milanowski, 2002; Kelly, Odden, Milanowski, & Heneman, 2000). We then shifted our attention to knowledge- and skill-based pay (KSBP) plans, an approach that provides teachers with base pay increases for the acquisition and demonstration of specific knowledge and skills thought to be necessary for improving student achievement.
Our initial research described a variety of experiments with KSBP plans (see Odden, Kelley, Heneman, & Milanowski, 2001). We found plans that were rewarding numerous knowledge and skills, including (a) additional licensure or certification, (b) participation in specific professional development activities, (c) National Board Certification, (d) mastery of specific skill blocks such as technology or authentic assessment, (e) leadership activities, and (f) teacher performance as measured by a standards-based teacher evaluation system. We also found districts experimenting with standards-based teacher evaluation without an intended pay link. As described below, in standards-based teacher evaluation systems, teachers' performance is evaluated against a set of standards that define a competency model of effective teaching. Such systems replace the traditional teacher evaluation system and seek to provide a more thorough description and accurate assessment of teacher performance. Findings from our research on some of these systems are the focus of this issue of CPRE Policy Briefs.
Describes reading-comprehension technique called "Questioning the Author," which focuses on teaching students to concentrate on building understanding of the ideas in the text. Provides examples of this reading-comprehension technique in geography and history. (PKP)
Since the beginning of the century, feedback interventions (FIs) produced negative--but largely ignored--effects on performance. A meta-analysis (607 effect sizes; 23,663 observations) suggests that FIs improved performance on average (
d = .41) but that over one-third of the FIs decreased performance. This finding cannot be explained by sampling error, feedback sign, or existing theories. The authors proposed a preliminary FI theory (FIT) and tested it with moderator analyses. The central assumption of FIT is that FIs change the locus of attention among 3 general and hierarchically organized levels of control: task learning, task motivation, and meta-tasks (including self-related) processes. The results suggest that FI effectiveness decreases as attention moves up the hierarchy closer to the self and away from the task. These findings are further moderated by task characteristics that are still poorly understood. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
In recent years, scholars have problematized terms used to describe instruction on teacher survey instruments. When scholars, observers, and teachers employ terms like discuss and investigate, they often mean to describe quite different events. This article problematizes another set of terms often found on survey instruments, those describing mathematical content. To do so, it examines terms such as geometry, number patterns, and ordering fractions for rates of agreement and disagreement between teachers and observers participating in a field pilot of an elementary mathematics daily log. Using interviews, written observations, and reflections on disagreements, this article also asks why disagreements occurred. Sources of disagreement included problems with instrument design, memory/perception, and, notably, differences in the way subject matter terms are used in different communities-university mathematicians, elementary teachers, and mathematics educators. Finally, implications of these sources of disagreement are explored.
Our concern in this paper is with the validity of educational tests when they are employed as critical measures of educational outcomes within a dynamic system. The problem of validity arises if an educational system adapts itself to the characteristics of the outcome measures. We introduce the concept of systemically valid tests as ones that induce curricular and instructional changes in education systems (and learning strategy changes in students) that foster the development of the cognitive traits that the tests are designed to measure. We analyze some general characteristics that contribute to or detract from a testing system's systemic validity, such as the use of direct rather than indirect assessment. We then apply these characteristics in developing a set of design principles for creating testing systems that are systemically valid. Finally, we provide an illustration of the proposed principles, by applying them to the design of a student assessment system. This design example addresses not only specifications for the tests, but also the means of teaching the process of assessment to users of the system.
Children come to school with an idea of narrative structure, but their knowledge of persuasive writing is less developed. Results of the National Assessment of Educational Progress indicate that students perform poorly on persuasive writing tasks. To what extent this is a result of a lack of logical thinking skills and to what extent it is a result of problems with instruction are not known. The present experiment was designed to identify the effects of instruction, grade level, and sex on students’ persuasive writing. Children in grades 4, 6, and 8 were instructed in writing with one of four instructional strategies. Their performance was then evaluated through analytic scoring of their efforts with standard writing prompts administered immediately after treatment and again 2 weeks later. The results showed that older children wrote better than younger ones; that girls wrote better than boys immediately after the study but not 2 weeks later; and that treatment effects approached statistical significance.
Research on teacher-child relationships, classroom environments, and teaching practices provided the rationale for constructing a system for observing and assessing emotional and instructional elements of quality in early childhood educational environments: the Classroom Assessment Scoring System (CLASS). The CLASS provides a framework for observing key dimensions of classroom processes, such as emotional and instructional support, that contribute to quality of the classroom setting from preschool through third grade. This article provides information about the development, field testing, and use of this instrument in prekindergarten. Data from a national sample of 224 prekindergarten classrooms in 6 states are presented to provide reliability and validity information. The full range of the scale was used for the majority of ratings. Ratings reflected generally positive impressions of the classroom environment and teacher-child interactions. Factor scores from the CLASS were related to the Early Childhood Environmental Rating Scale (ECERS) total score and most strongly related to the ECERS interactions and language-reasoning subscales. Implications for policy and professional development from prekindergarten to third grade are discussed.
THIS STUDY investigated the implications of signifying, a form of social discourse in the African-American community, as a scaffold for teaching skills in literary interpretation. This investigation is related to the larger question of the efficacy of culturally sensitive instruction. The major premise on which the hypotheses of this study are based is the proposal that African American adolescents who are skilled in signifying use certain strategies to process signifying dialogue. These strategies are comparable to those that expert readers use in order to construct inferences about figurative passages in narrative texts. In order to apply this premise, an instructional unit was designed aimed at helping students bring to a conscious level the strategies it is presumed they use tacitly in social discourse. This approach is offered as a model of cognitive apprenticing based on cultural foundations. Analyses are presented of how the cultural practice links to heuristic strategies that experts use in a specific domain, as well as how instructors modeled, coached, and scaffolded students.
There are fierce debates over the best way to prepare teachers. Some argue that easing entry into teaching is necessary to attract strong candidates, whereas others argue that investing in high quality teacher preparation is the most promising approach. Most agree, however, that we lack a strong research basis for understanding how to prepare teachers. This article is one of the first to estimate the effects of features of teachers' preparation on teachers' value added to student test score performance. Our results indicate variation across preparation programs in the average effectiveness of the teachers they are supplying to New York City schools. In particular, preparation directly linked to practice appears to benefit teachers in their 1st year. (Contains 6 notes, 5 tables, and 1 figure.)
The TESOL Quarterly welcomes evaluative reviews of publications relevant to TESOL professionals. In addition to textbooks and reference materials, these include computer and video software, testing instruments, and other forms of nonprint materials.
In the last quarter-century and especially the last decade, testing and accountability have come to dominate education policy at the state and national levels. The common concern about the effects of such testing is that it reshapes teaching in the classroom. But such claims do not look at the evidence of deeper classroom structures (the mix of teacher-centered and student-centered practices) in historical context. This article extends historical research in How Teachers Taught (Cuban, 1993) to the present in three metropolitan school districts. While testing and accountability have become more obvious concerns of teachers, the hybridized classroom environment documented in How Teachers Taught have become more pervasive. This article documents this continuing ubiquity and addresses the apparent inconsistency between evidence of a hybridized classroom environment and the unintended consequences of testing and accountability.
This nonexperimental, longitudinal field study examines the extent to which variation in observed classroom supports (quality of emotional and instructional interactions and amount of exposure to literacy and math activities) predicts trajectories of achievement in reading and math from 54 months to fifth grade. Growth mixture modeling detected two latent classes of readers: fast readers whose skills developed rapidly and leveled off, and a typical group for which reading growth was somewhat less rapid. Only one latent class was identified for math achievement. For reading, there were small positive associations between observed emotional quality of teacher-child interactions and growth. Growth in math achievement showed small positive relations with observed emotional interactions and exposure to math activities. There was a significant interaction between quality and quantity of instruction for reading such that at higher levels of emotional quality there was less of a negative association between amount of literacy exposure and reading growth.
In recent years, scholars have problematized terms used to describe instruction on teacher survey instruments. When scholars, observers, and teachers employ terms like discuss and investigate, they often mean to describe quite different events. This article problematizes another set of terms often found on survey instruments, those describing mathematical content. To do so, it examines terms such as geometry, number patterns, and ordering fractions for rates of agreement and disagreement between teachers and observers participating in a field pilot of an elementary mathematics daily log. Using interviews, written observations, and reflections on disagreements, this article also asks why disagreements occurred. Sources of disagreement included problems with instrument design, memory/perception, and, notably, differences in the way subject matter terms are used in different communities—university mathematicians, elementary teachers, and mathematics educators. Finally, implications of these sources of disagreement are explored.
This paper discusses conceptual and methodological issues that a-rise when educational researchers use data from large-scale, survey research to examine the effects of teachers and teaching on student achievement. Using data from Prospects: The Congressionally Mandated Study of Educational Growth and Opportunity 1991-1994, we show that researchers' use of different statistical models has led to widely varying interpretations about the overall magnitude of teacher effects on student achievement. However, we conclude that in well-specified models of academic growth, teacher effects on elementary school students' growth in reading and mathematics achievement are substantial (with d-type effect sizes ranging from .72 to .85). We also conclude that various characteristics of teachers and their teaching account for these effects, including variation among teachers in professional preparation and content knowledge, use of teaching routines, and Patterns of content coverage, with effect sizes for variables measuring these characteristics of teachers and their teaching showing d-type effect sizes in the range of .10. The paper concludes with an assessment of the current state of the art in large-scale, survey research on teaching. Here, we conclude that survey researchers must simultaneously improve their measures of instruction while paying careful attention to issues of causal inference.
The study reported in this article focused on the preparation of elementary preservice teachers to teach reading and on their first 3 years of teaching in schools. Graduates of 8 programs judged as "excellent" by an expert review panel participated in this study. The research was guided by 2 questions: (1) What effects do participation in and completion of an excellent reading teacher education program have on the experiences of teachers as they enter schools? and (2) How does teachers' preparation relate to their teaching practices? We used quantitative and qualitative research methods to explore these questions. The research design was quasi-experimental, with the teacher education program considered as the intervening variable. Comparison groups for graduates of the excellent programs included same-school, highly experienced teachers as well as same-school, same-years experienced teachers. Data were collected over 3 years. Results suggested that participation in a high-quality teacher preparation program had a positive influence on the transition of teachers entering the profession and on the adoption of effective teaching practices by these teachers. Graduates of the excellent programs were more effective than teachers in the comparison groups in creating and engaging their students with a high-quality literacy environment.
We report on the development and validation of the TEX-IN3, a research tool and protocol designed to assess the quality of the classroom literacy environment. The TEX-IN3 is described in terms of its theoretical base as well as its practical applications. The TEX-IN3 includes three basic components: a text inventory, a text “in-use” observation, and a series of text interviews. The validation of the TEX-IN3 is first reported in terms of its content, development, and focus. Next, a validation study examining the relationship between the TEX-IN3 and student growth in reading comprehension is described. The findings from this study offer support for all three components of the instrument as associated with student achievement growth.
In this article, we present the position that students with special needs, regardless of causation, share the same basic needs for literacy instruction. What sets special needs learners apart from higher performing learners is that systematic, explicit instruction appears to be critical rather than merely beneficial and that special needs students require more instructional time and opportunity to practice reading. The need for greater explicitness and time intensity does not, however, change the content of what composes good literacy instruction. We take the position that general education should maintain primary responsibility for literacy education for all children, with special educators and reading specialists providing instructional support and, if necessary, additional instructional intensity and practice opportunities. Obstacles to providing quality literacy instruction are examined, and a school-based case study in which these obstacles have been largely overcome are presented.
numbers of students in secondary schools who do not read well. In response, com-mitted and well-meaning educators are increasingly advocating remedial reading courses for struggling adolescent readers. In this article, Cynthia Greenleaf, Ruth Schoenbach, Christine Cziko, and Faye Mueller offer an alternative vision to re-medial reading instruction. The authors describe an instructional framework, Reading Apprenticeship, that is based on a socially and cognitively complex conception of literacy, and examine an Academic Literacy course based on this framework. They demonstrate that academically underperforming students partic-ipating in the Academic Literacy course gained on average what is normally two years of reading growth within one academic year on a standardized test of read-ing comprehension. They argue for investing resources and effort into demystifyi-ng academic reading for their students through ongoing, collaborative inquiry into reading and texts, while providing students with protected time for reading and access to a variety of attractive texts linked to their curriculum. This ap-proach can move students beyond the "literacy ceiling" to increased understand-ing, motivation, opportunity, and agency as readers and learners. These findings challenge the current policy push for remedial reading programs for poor readers, and invite further research into what factors create successful reading instruction programs for secondary school students.
Lisa Delpit uses the debate over process-oriented versus skills-oriented writing instruction as the starting-off point to examine the "culture of power" that exists in society in general and in the educational environment in particular. She analyzes five complex rules of power that explicitly and implicitly influence the debate over meeting the educational needs of Black and poor students on all levels. Delpit concludes that teachers must teach all students the explicit and implicit rules of power as a first step toward a more just society. This article is an edited version of a speech presented at the Ninth Annual Ethnography in Education Research Forum, University of Pennsylvania, Philadelphia, Pennsylvania, February 5-6, 1988.
This article reviews the state of multiparameter shrinkage estimators with emphasis on the empirical Bayes viewpoint, particularly in the case of parametric prior distributions. Some successful applications of major importance are considered. Recent results concerning estimates of error and confidence intervals are described and illustrated with data.
Our concern in this paper is with the validity of educational tests when they are employed as critical measures of educational outcomes within a dynamic system. The problem of validity arises if an educational system adapts itself to the characteristics of the outcome measures. We introduce the concept of systemically valid tests as ones that induce curricular and instructional changes in education systems (and learning strategy changes in students) that foster the development of the cognitive traits that the tests are designed to measure. We analyze some general characteristics that contribute to or detract from a testing system's systemic validity, such as the use of direct rather than indirect assessment. We then apply these characteristics in developing a set of design principles for creating testing systems that are systemically valid. Finally, we provide an illustration of the proposed principles, by applying them to the design of a student assessment system. This design example addresses not only specifications for the tests, but also the means of teaching the process of assessment to users of the system.
Ideas like "understanding," "authenticity," and "community" are central in current debates about curriculum, instruction, and assessment. Many believe that teaching and learning would be improved if classrooms were organized to engage students in authentic tasks, guided by teachers with deep disciplinary understandings. Students would conjecture, experiment, and make arguments; they would frame and solve problems; and they would read, write, and create things that mattered to them. This article examines the challenge of creating classroom practices in the spirit of these ideals. With a window on her own teaching of elementary school mathematics, the author presents three dilemmas-of content, discourse, and community-that arise in trying to teach in ways that are, in Bruner's terms, "intellectually honest." These dilemmas arise reasonably from competing and worth-while aims and from the uncertainties inherent in striving to attain them. The article traces and explores the author's framing of and response t...
The utility of value-added estimates of teachers' effects on student test scores depends on whether they can distinguish between high- and low-productivity teachers and predict future teacher performance. This article studies the year-to-year variability in value-added measures for elementary and middle school mathematics teachers from five large Florida school districts. We find year-to-year correlations in value-added measures in the range of 0.2–0.5 for elementary school and 0.3–0.7 for middle school teachers. Much of the variation in measured teacher performance (roughly 30–60 percent) is due to sampling error from “noise” in student test scores. Persistent teacher effects account for about 50 percent of the variation not due to noise for elementary teachers and about 70 percent for middle school teachers. The remaining variance is due to teacher-level time-varying factors, but little of it is explained by observed teacher characteristics. Averaging estimates from two years greatly improves their ability to predict future performance. © 2009 American Education Finance Association
This article considers potential impediments to the estimation of teacher quality caused primarily by the purposeful behavior of families, administrators, and teachers. The discussion highlights the benefits of accounting for student and school differences through a value-added modeling approach that incorporates a student's history of family, school, and community influences. Unfortunately, the value-added framework does not address all potential impediments to the consistent estimation of teacher quality of instruction, even when focused solely on within-school differences. Of particular importance is the nonrandom assignment of students to classrooms on the basis of unobservable characteristics and parental and school responses to teacher quality. Evidence from studies that take extensive measures to mitigate the influence of purposeful classroom assignment nonetheless supports the existence of substantial within-school variation in the quality of instruction.
Annual student testing may make it possible to measure the contributions to student achievement made by individual teachers. But would these “teacher value-added” measures help to improve student achievement? I consider the statistical validity, purposes, and costs of teacher value-added policies. Many of the key assumptions of teacher value added are rejected by empirical evidence. However, the assumption violations may not be severe, and value-added measures still seem to contain useful information. I also compare teacher value-added accountability with three main policy alternatives: teacher credentials, school value-added accountability, and formative uses of test data. I argue that using teacher value-added measures is likely to increase student achievement more efficiently than a teacher credentials-only strategy but may not be the most cost-effective policy overall. Resolving this issue will require a new research and policy agenda that goes beyond analysis of assumptions and statistical properties and focuses on the effects of actual policy alternatives. © 2009 American Education Finance Association
This book is about how language is used in the context of schooling. It demonstrates that the variety of English expected at school differs from the interactional language that students use for social purposes outside of school, and provides a linguistic analysis of the challenges of the school curriculum, particularly for non-native speakers of English, speakers of non-standard dialects, and students who have little exposure to academic language outside of schools. The Language of Schooling: A Functional Linguistics Perspective builds on current sociolinguistic and discourse-analytic studies of language in school, but adds a new dimension--the framework of functional linguistic analysis. This framework focuses not just on the structure of words and sentences, but on how texts are constructed--how particular grammatical choices create meanings in the different kinds of texts students are asked to read and write at school. The Language of Schooling: A Functional Linguistics Perspective * provides a functional description of the kinds of texts students are expected to read and write at school; * relates research from other sociolinguistic and language development perspectives to research from the systemic functional linguistics perspective; * focuses on the increasing linguistic demands of contexts of advanced literacy (middle school through college); * analyzes the genres typically encountered at school, with extensive description of the grammatical features of the expository essay, a gatekeeping genre for secondary school graduates; * reviews the grammatical features of disciplinary genres in science and history; and * argues for more explicit attention to language in teaching all subjects, with a particular focus on what is needed for the development of critical literacy. This book will enable researchers and students of language in education to recognize how the grammatical and discourse features of the language of schooling construct the content areas, role relationships, and purposes and expectations of schools. It also will enable them to better understand the nature of language itself and how it emerges from and helps to maintain social structures and institutions, and to apply these understandings to creating classroom environments that build on the strengths students bring to school. © 2004 by Lawrence Erlbaum Associates, Inc. All rights reserved.
This document includes the final data collection and scoring tools created by the "Scoop" project, a five-year project funded through the Center for Evaluation, Standards,and Student Testing (CRESST), to develop an alternative approach for characterizing classroom practice. The goal of the project was to use artifacts and related materials to represent classroom practice well enough that a person unfamiliar with the teacher or the lessons can make valid judgments about selected features of practice solely on the basis of those materials. The artifacts and other materials were collected in a binder called the Scoop Notebook. Thus, the project sought to answer the question, "Can accurate judgments about reform-oriented instructional practice be made based on the classroom artifacts and teacher reflections assembled in the Scoop Notebook?" This document describes the Scoop Notebook and the rating guides, gives instructions for assembling the materials and explaining the process to teachers, and discusses two potential uses of the Scoop Notebook--as a tool to characterize classroom practice or as a tool for teacher professional development. The appendices present the final versions of the Scoop Notebook and rating guide for both mathematics and science. (Contains 3 footnotes.)
Reading and social studies instruction in 39 third- through sixth-grade classrooms in central Illinois was observed in a study designed to determine whether elementary school classrooms provide comprehension instruction. Among the findings were that practically no comprehension instruction was seen but that comprehension assessment, usually carried on through interrogation, was common; that other kinds of reading instruction were not seen with any frequency; that in addition to being interrogators, teachers also appeared to be assignment-givers, spending a large part of the observed periods on giving and checking assignments; that sizeable amounts of time went to activities categorized as "transition" and "noninstruction"; and that teachers did not see the social studies period as a time to improve children's comprehension abilities, but instead were concerned about covering content and having children master facts. (The report discusses definitions and examples of comprehension instruction, notes differences between comprehension instruction and comprehension assessment, explains the categories used for teacher and student behavior, offers suggestions for future research, and reports the findings of the study in descriptive and tabular form.) (GW)
Studies of preschool indicate that: (1) valid and reliable measures of the environment are possible; (2) environments vary across programs; and (3) there is a relationship between the environment and development of children in those environments. The Early Childhood Environment Rating Scale shows promise for both research and training. (Author/LC)