Analyzing Differentiation in the Classroom: Using the COS-R

Abstract

The use of a classroom observation tool to monitor differentiation strategies is described, and relevant research findings using the form are reported. The advantages for using this approach to document differentiation are discussed as are the reasons teachers may question its intent. Applications for practice include its use as a self-assessment tool to guide teaching improvement in differentiation.

Full text

http://gct.sagepub.com/
Gifted Child Today
http://gct.sagepub.com/content/35/1/42
The online version of this article can be found at:
DOI: 10.1177/1076217511427431
2012 35: 42Gifted Child Today
Joyce VanTassel-Baska
Analyzing Differentiation in the Classroom: Using the COS-R
Published by:
http://www.sagepublications.com
can be found at:Gifted Child TodayAdditional services and information for
http://gct.sagepub.com/cgi/alertsEmail Alerts:
http://gct.sagepub.com/subscriptionsSubscriptions:
http://www.sagepub.com/journalsReprints.navReprints:
http://www.sagepub.com/journalsPermissions.navPermissions:
http://gct.sagepub.com/content/35/1/42.refs.htmlCitations:
What is This?
- Feb 28, 2012Version of Record >>
by guest on October 11, 2013gct.sagepub.comDownloaded from by guest on October 11, 2013gct.sagepub.comDownloaded from by guest on October 11, 2013gct.sagepub.comDownloaded from by guest on October 11, 2013gct.sagepub.comDownloaded from by guest on October 11, 2013gct.sagepub.comDownloaded from by guest on October 11, 2013gct.sagepub.comDownloaded from by guest on October 11, 2013gct.sagepub.comDownloaded from by guest on October 11, 2013gct.sagepub.comDownloaded from

43
vol. 35 ■ no. 1
GIFTED CHILD TODAY
DOI: 10.1177/1076217511427431. From
1
College of William and Mary. Address correspondence to: Joyce VanTassel-Baska, Center for Gifted Education, College of William and Mary, 427
Scotland Street, Williamsburg, VA 23185; email: jlvant@wm.edu.
For reprints and permissions queries, please visit SAGE’s Web site at http://www.sagepub.com/journalsPermission.nav.
Copyright © 2012 The Author(s)
Abstract: The use of a classroom observation tool
to monitor differentiation strategies is described, and
relevant research findings using the form are reported.
The advantages for using this approach to document
differentiation are discussed as are the reasons teachers may
question its intent. Applications for practice include its use
as a self-assessment tool to guide teaching improvement in
differentiation.
Keywords: teacher observation, differentiation, monitoring,
classroom practice, critical thinking, research, problem solving,
creative thinking
O
ver the last several years, there has been considerable
evidence from different areas suggesting that how
teachers behave in the classroom and the instructional
approaches they use significantly affect the degree that
students learn. Sanders and Rivers (1996) reported the effects
of ineffective teachers over 3 years as having a depressed
effect on student achievement in math by as much as 54%
regardless of the ability of the learner. Wenglinsky (2000) found
positive effects of using key practices such as critical thinking
and metacognition on student learning in math and science
across elementary and middle school levels. The literature in
gifted education suggests that teacher behavior is the link to
differentiated programs and services for this special population.
Studies consistently report that regular classrooms have very
limited differentiated activities (Westberg, Archambault, Dobyns,
& Salvin, 1993; Westberg & Daoust, 2003). Moreover, gifted and
talented teacher behaviors are not systematically monitored.
Classroom differentiation is highly dependent on positive
teacher behavioral change in key areas. A study of math and
science programs found that teachers will use strategies linked
to content that show results with students (Kennedy, 1999).
Collegiality and support have also been found to be necessary
for change to occur (Garet, Porter, Desimone, Birman, & Yoon,
2001). Another study showed that the use of higher level
teaching behavior took 2 or more years of intensive training to
demonstrate results (Borko, Mayfield, Marion, Flexer, & Cumbo,
1993). Based on these findings, it is clear that attention to
classroom-level instruction must be carefully monitored for the
results used to improve teaching.
Yet, use of classroom monitoring and evaluation, alone or
disconnected from ongoing professional development, appears
to be unsuccessful as a catalyst for teacher change. A recent
study of teacher evaluation practices in three districts in
Wisconsin (Kimball, 2002) found that few teachers reported
substantial changes in their instructional practice as a result of
their evaluation experiences, and the large majority of teachers
did not see the evaluation process as an incentive to seek out
professional development opportunities.
The field of gifted education is recognized for advancing
the introduction of innovative instructional practices
into the classroom, such as inquiry learning, critical and
creative thinking skills, higher order questioning strategies,
metacognition, and the use of rich and varied curricular
materials, rather than sole reliance on textbooks (Tomlinson &
Callahan, 1992). In the last decade, the introduction of content-
based curriculum tied to state and national standards and
evaluated on student learning gains (VanTassel-Baska, Bass,
Ries, Poland, & Avery, 1998; VanTassel-Baska, Zuo, Avery, &
Little, 2002) has again positioned the field in the forefront of
curriculum innovation and differentiation, with the emphasis on
“going through the standards and not around them” to achieve
instructional impact.
In spite of such advances, there is little evidence that gifted
education programs systematically assess student gains using
appropriate learning measures (Avery & VanTassel-Baska,
2002). In the absence of student impact data, evaluators must
often rely on the quality of the instructional experience as
a proxy method for investigating program effectiveness. In
fact, teacher effectiveness has been shown to be the main
FEATUR ES
Analyzing Differentiation in the Classroom
Using the COS-R
Joyce VanTassel-Baska, EdD
1

44
GIFTED CHILD TODAY January 2012
determinant of student progress (W. L. Sanders & Horn,
1998). Classroom observation provides a nexus between the
input variables of the teacher and his or her students and
the process of instruction itself, a process that combines
instructional intent (goals and objectives), curriculum resources
and materials, instructional and assessment strategies, and
classroom management skills within a delimited period of
time. In meeting the needs of the gifted learner, the observer
must focus on three dimensions of
practice: good teaching in general, key
elements of educational reform, and
differentiation for high-ability learners.
There are certain assumptions made
about the Teaching Act that underlay
the use of any observation tool,
especially one like the Classroom
Observation Scale–Revised (COS-
R), which is extensive and in depth
in respect to behaviors (Figure 1).
Teaching is a complex social activity
requiring the capacity to split attention
by student, by area of the room, and by activity. It requires
making multiple decisions during a teaching episode by
instructional regulation, strategies, use of time, and lesson
emphasis. Moreover, teachers must think in complex ways to
implement many behaviors simultaneously. For example, one
type of thinking that teachers must use relates to following the
lesson plan, responding to pacing, modulating student–teacher
interaction, and ensuring a variety of stimuli. A second type
of thinking required of teachers is awareness of teaching to
multiple objectives, such as teaching a concept at the same time
as they are teaching content and skills, and at the same time
they are attending to the teaching of group dynamics.
A third way of thinking involves orchestrating feedback
strategies and responses, flexibility in grouping patterns, and
questioning and activity choices. Planning, monitoring, and
assessing both group and individual learning are another level
of thinking that teachers must orchestrate in their daily practice.
How does the current lesson relate to yesterday’s and how will
it link to tomorrow’s?
Teaching has traditionally been a solitary activity; thus,
teachers are not socialized to having external observations
of their work and oftentimes are uncomfortable with the
observation process. Yet, improvement in teaching clearly
requires a change in teacher behaviors that promote learning in
students. Such improvement appears to imply the use of higher
order thinking, problem solving, and metacognitive approaches.
To ensure that teachers are using such strategies, some form of
monitoring teacher behaviors must occur.
Goals for the COS-R
Classroom observation is a seminal part of understanding
positive change in education. It affords an opportunity to
access the actual instructional experience that is at the heart of
teaching and learning. It provides a nexus between the input
variables of the teacher and his or her students and the process
of instruction itself, a process that combines instructional
intent (goals and objectives), curriculum resources and
materials, instructional strategies, and classroom management
skills within a delimited unit of time. It is the one part of
professional development that allows the critical pieces of
teacher knowledge and skills to come together in an authentic
opportunity to gain insight about the
quality of the learning experiences that
are delivered.
One way of thinking about a
classroom observation component
is to see it as a performance-based
assessment of the teacher within the
context of the learning environment.
It affords many of the features of
performance-based assessment with
the teacher, rather than the student, as
the unit of focus. For instance, it is a
relatively open-ended experience, with
teachers exercising much control over the selection of the lesson
to be taught. It allows for the demonstration of complex and
higher order behaviors, recognizing that good teaching derives
from a sophisticated set of skills that unfold in an integrated
way. It also allows for self-assessment, providing a metacognitive
dimension to the experience. Most importantly, by using a
structured form, it provides a benchmark against which the
teaching process can be assessed based on expectations derived
from best practice in a given field. However, it is important to
distinguish between program and teacher evaluation in terms
of the parameters used in the classroom observation process.
The focus of program evaluation targets the collective whole.
Teacher behavior is sampled through the classroom observation
process, and the intent is to allow teachers to prepare for the
observation period to reduce the level of threat that permeates
this current school climate of accountability. By aggregating data
across classrooms, a snapshot of instructional practice is created
that helps inform our understanding of program quality. Direct
observation is also an important component of a teacher (or
staff) evaluation process, but deeper sampling of the individual
being assessed must occur when used for this purpose.
Also, teacher evaluation should involve some opportunity for
unplanned visits to the classroom to assess the consistency of
instructional performance. While the two uses of classroom
observation (program and teacher evaluation) can complement
one another, it is important to acknowledge that the limited
sampling template used for program evaluation is not sufficient
to make inferences regarding individual staff performance tied to
retention or promotion decisions.
Applications of the COS-R
If classroom practice for the high-ability learner requires
“differentiated” services, then how is this criterion translated
“
Classroom
observation is
a seminal part of
understanding
positive change in
education.”

45
vol. 35 ■ no. 1
GIFTED CHILD TODAY
Figure 1. The Classroom Observation Scale–Revised

46
GIFTED CHILD TODAY January 2012
Figure 1. (continued)

47
vol. 35 ■ no. 1
GIFTED CHILD TODAY
into expectations for teacher performance and accountability?
What constitutes “practice” in classrooms serving gifted
learners? The COS-R provides a mechanism for using
quantitative parameters to answer these questions. We cannot
afford to ignore teacher behaviors that theory and research
have found to contribute to improved teaching effectiveness.
The COS-R is one way to assess individual teacher performance
in response to high-ability learners.
The process of COS-R development has involved several
stages and forms occurring over more than a decade
(VanTassel-Baska, 2004). At the first stage, the project team
reviewed extant literature on effective teaching, differentiated
instruction for gifted learners, educational reform and change,
and professional development research. Based on expectations
derived from best practices in mainstream and gifted
education classrooms, the form was developed to be used in
all classrooms and in all subject areas. Only the most critical
behaviors for general teaching and differentiation features
culled from research-based evidence of effective classroom-
based instructional behaviors were included. These focus
on the use of strategies that promote student learning and
growth especially in the area of higher order thinking, problem
solving, and metacognition. Clusters were created based on
predominant teaching behaviors within each category. In its
final iteration, the COS-R comprised a total of 25 items in six
clusters.
Technical Adequacy
The analyses in three studies showed that, overall, the scale
was highly reliable (a = .91 to .93). The subscale reliability for
all the clusters averaged above .70. Based on a content-validity
assessment by four scholars in gifted education, the scale
achieved an intraclass coefficient of .98. Interrater reliability was
assessed in four studies, with results suggesting a range of .87
to .89 across trained raters.
Research suggests that although teachers of the gifted
appeared strong in many categories of good teaching, they
fall short in many others that examine differentiation practices
(VanTassel-Baska, 2004). For instance, higher level thinking
models were less in use than might be expected in classrooms
when gifted learners form a critical mass of the student body.
Very little emphasis was placed on accelerative strategies in
gifted programs. Rate differences between gifted learners were
not routinely attended to by classroom teachers, even those
trained in gifted education. Models that explore different types
of thinking and problem-solving behavior were not routinely
used in gifted program classrooms. The most prevalent
problem-solving behavior observed was brainstorming, many
times used in isolation of more fulsome models that examined
a broader array of problem-solving behavior. Little evidence of
the use of metacognition in gifted classrooms was observed.
Moreover, little emphasis of out-of-class learning was found in
observed teachers of the gifted classrooms as judged by their
lack of provision for extended activities. Individualizing for
differences within the gifted population, critical thinking,
and problem solving were found to be underused in most
observed classrooms. Core teaching behaviors like lesson
planning and clarity in providing directions were more
routinely observed. Curriculum reform elements were in
evidence unevenly, with greater prevalence of concept teaching
than other behaviors.
Thus, the COS-R provides direct evidence of the need
for specific emphasis in developing programs and may
be effectively used as a planning tool for that purpose.
Professional development plans must take into account the
evolving skills of teachers such that workshops and seminars
are more tightly focused on needed skills as opposed to
desired areas of interests. Gifted programs need to consider
the sophistication of using higher level skills effectively and
the need for teachers to develop these skills over time through
appropriate methods (VanTassel-Baska & Little, 2011).
The COS-R may also prove useful in charting overall growth
in desirable teacher behaviors by school or district in respect
to the new National Association for Gifted Children standards
for curriculum, environment, assessment, and professional
development (Sulak & Johnsen, in press). Diagnoses of
individual teacher skills needed to work effectively with gifted
learners can also easily occur through the use of the COS-
R. Videotape analysis of teaching can employ self-assessment
using the instrument as well as external observers.
Implications of Using an Observation Model
for Monitoring Differentiation
For many teachers, there is still a hesitancy to open
their classrooms to observation, whether from building
administrators or outside trained observers. Yet, for
differentiation to progress in a meaningful way, teachers
must be willing to reveal their practice and learn from the
experience. Putting them into situations where they can
converse with observers before and after a lesson, where
they can self-assess, and where there is a critical mass of
teachers experiencing the same process makes the experience
more useful and points the way to improvement in
strategy use.
Another implication of using such a tool is a potential conflict
with existing teacher evaluation forms that are widely used
and accepted by schools and teachers as a basis for making
judgments about teacher competency. Because the use of
successful differentiation techniques goes beyond the level of
competency expected on many of those forms, the COS-R and
instruments like it can cause confusion in schools about how
it relates to the districtwide forms and models used. It is often
advisable to align both the forms and the processes used to
clarify purpose and intent. Because differentiation strategies
require higher level skills of the teacher, using the form at a
second level of review may be warranted.
Finally, the COS-R requires multiple use with the same group
of teachers. Observations should be three to four per year to

48
GIFTED CHILD TODAY January 2012
get a good snapshot of classroom practice. Multiple use also
allows for teacher improvement to show as discussions can
be held to target particular differentiation strategies to be in
evidence. For example, teachers may plan to activate certain
research strategies at a particular time and invite observers,
deliberately, to view such lessons. Feedback to teachers, then,
can be based on stated desire rather than an inflexible schedule
of visitation.
Teaching is a performance art. Therefore, the test of
success is demonstrating mastery of relevant techniques. Use
of monitoring approaches, including coaching, structured
observation, and videotape analysis, increases the chance that
teachers will improve in important ways in behaviors that
optimize learning for our best students.
Conflict of Interest
The author(s) declare no potential conflicts of interest with
respect to the authorship and/or publication of this article.
References
Avery, L. D., & VanTassel-Baska, J. (2002). The impact of gifted education
evaluation at state and local levels: Translating results into action.
Journal for the Education of the Gifted, 25, 153 -176.
Borko, H., Mayfield, V., Marion, S. F., Flexer, R. J., & Cumbo, K. (1993).
Teachers developing ideas and practices about mathematics
performance assessment: Successes, stumbling blocks, and
implications for professional development. Teaching and Teacher
Education, 13, 259-278. Retrieved from www.cse.ucla.edu/products/
Reports/TECH423.pdf
Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S.
(2001). What makes professional development effective: Results
from a national sample of teachers. American Research Journal, 38,
915-945.
Kennedy, M. (1999). Form and substance in mathematics and science
professional development. NISE Brief, 3(2), 1-7.
Kimball, S. M. (2002). Analysis of feedback, enabling conditions and
fairness perceptions of teachers in three school districts with new
standards-based evaluation systems. Journal of Personnel Evaluation
in Education, 16, 241-268.
Sanders, W. I., & Rivers, J. C. (1996). Cumulative and residual effects of
teachers on future students’ academic achievement. Knoxville, TN:
University of Tennessee Value-Added Research and Assessment
Center.
Sanders, W. L., & Horn, S. P. (1998). Research findings from the Tennessee
value-added assessment system (TVASS) database: Implications for
educational research and evaluation. Journal of Personnel Evaluation
in Education, 12, 247-256.
Sulak, T. N., & Johnsen, S. K. (2012). Assessments for measuring student
outcomes. In S. K. Johnsen (Ed.), Using the NAGC pre-K-grade 12
gifted programming standards (p. 283-306). Waco, TX: Prufrock
Press.
Tomlinson, C. A., & Callahan, C. (1992). Contributions of gifted education
to general education in a time of change. Gifted Child Quarterly, 36,
183-189.
VanTassel-Baska, J. (2004). Assessing classroom practice: The use of a
structured observation form. In J. VanTassel-Baska & A. Feng (Eds.),
Designing and utilizing evaluation for gifted program improvement
(pp. 87-108). Waco, TX: Prufrock Press.
VanTassel-Baska, J., Bass, G. M., Ries, R. R., Poland, D., & Avery, L. D.
(1998). A national pilot study of science curriculum effectiveness for
high ability learners. Gifted Child Quarterly, 42, 200-211.
VanTassel-Baska, J., & Little, C. (Eds.). (2011). Content based curriculum
for high ability learners. Waco, TX: Prufrock Press.
VanTassel-Baska, J., Zuo, L., Avery, L. D., & Little, C. A. (2002). A
curriculum study of gifted student learning in the language arts.
Gifted Child Quarterly, 46, 30-44.
Wenglinsky, H. (2000). How teaching matters: Bringing the classroom back
into discussions of teacher quality. Princeton, NJ: Educational Testing
Service.
Westberg, K. L., Archambault, F. X., Dobyns, S. M., & Salvin, T. J. (1993).
The classroom practices observation study. Journal for the Education
of the Gifted, 16, 120 -146.
Westberg, K. L., & Daoust, M. E. (2003). The results of the replication of
the classroom practices survey replication in two states. The National
Research Center on Gifted and Talented Newsletter, Fall, 3-8.
Bio
Joyce VanTassel-Baska, EdD, is the Smith professor emerita at the
College of William and Mary in Virginia where she developed a
graduate program and a research and development center in
gifted education. Formerly, she initiated and directed the Center
for Talent Development at Northwestern University. She has also
served as the state director of gifted programs for Illinois; as a
regional director of a gifted service center in the Chicago area;
as coordinator of gifted programs for the Toledo, Ohio public
school system; and as a teacher of gifted high school students in
English and Latin. She has published widely, including 27 books
and more than 500 refereed journal articles, book chapters, and
scholarly reports. Her major research interests are on the talent
development process and effective curricular interventions with
the gifted.