Content uploaded by Deborah N. Filbin
Author content
All content in this area was uploaded by Deborah N. Filbin on Sep 13, 2020
Content may be subject to copyright.
ABSTRACT
ART ASSESSMENT POLICY AND PRACTICE AT THE HIGH SCHOOL LEVEL:
VALIDITY, RELIABILITY, AND RESISTANCE
Deborah N. Filbin, Ph.D.
Northern Illinois University, 2020
School of Art and Design
Douglas Boughton and Kerry Freedman, Co-Directors
National education reform policies that have increased reliance on standardized testing in
subjects like reading and math as a way to judge the quality of a teacher’s performance have
created challenges for educators in visual art and design when they are required to provide data
about student growth. Art teachers who utilize alternate forms of assessment to judge the quality
of student artwork as evidence of learning can potentially be in a precarious position because of
underlying assumptions that these types of qualitative assessments lack validity. In this context,
when compared to colleagues in traditionally tested disciplines, art educators face unique
challenges proving their assessments are both valid and reliable. Framed in critical pedagogy, a
mixed-methods study was conducted in the state of Illinois to investigate the kinds of assessment
strategies high school art teachers found useful in their classrooms to measure student learning
and whether their methods differed from the types of assessments their administrators expected.
This study brings attention to the important aspects of assessment and how policy can shape
teachers’ practice.
This study provided unique insight into Illinois art teachers’ experiences with the current
state teacher evaluation policy PERA (Performance Evaluation Reform Act). Participants from
throughout the state were surveyed about their understanding of assessment, validity, reliability,
and professional development. To elaborate and contextualize the findings, face-to- face
interviews were conducted with eight participants to obtain a deeper understanding of teacher’s
actual experiences in the classroom and elaborate upon the role educational policy played in
assisting them to meet requirements for their performance evaluation. Assessment at the high
school level can present unique challenges when compared to other disciplines because of the
complexity of qualitative judgments teachers must make about their students’ work. Applying
qualitative assessment methods were particularly troublesome for participants because their
methods were not like traditional right-wrong answer choice tests; this created underlying
mistrust of art teachers’ data that was derived from professional judgement on qualitative
assessments. Deepening the challenge art teachers faced when assessing student art performance,
the participants described a lack of professional development specifically for assessment in art.
They also struggled with an absence of professional development to establish the validity and
reliability of their assessments and sometimes received inappropriate direction about student
performance data collection because of a lack of understanding by administrators or supervisors
about how student artwork should be judged. In an age of test-based accountability, professional
development for teachers, administrators, and pre-service educators aimed specifically for art
and design assessment is essential for practitioners in public schools.
Additionally, it was found the socio-economic status of teacher participants’ schools was
related to the kinds of data they were asked to collect and the kinds of resources they had
available within their departments thus indicating a lack of equitable access to quality art
education throughout the state. In response to administrative requests that participants felt were
inappropriate to their discipline, multiple forms of resistance were exhibited including covert,
overt, and passive compliance as a way to help them cope with what many participants felt were
overwhelming obstacles to teaching a quality visual art curriculum.
2
NORTHERN ILLINOIS UNIVERSITY
DEKALB, ILLINOIS
MAY 2020
ART ASSESSMENT POLICY AND PRACTICE AT THE HIGH SCHOOL LEVEL:
VALIDITY, RELIABILITY, AND RESISTANCE
BY
DEBORAH N. FILBIN
© 2020 Deborah N. Filbin
A DISSERTATION SUBMITTED TO THE GRADUATE SCHOOL
IN PARTIAL FULLFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE
DOCTOR OF PHILOSOPHY
SCHOOL OF ART AND DESIGN
Doctoral Co-Directors:
Douglas Boughton and Kerry Freedman
3
BACKGROUND TO THE STUDY
I have been teaching art and design at the high school level for over 20 years. Becoming a
public-school teacher is the most rewarding choice I could have ever made. However, throughout
my career I have perceived a slow and steady erosion of morale, loss of self-efficacy, and a
reduction in job satisfaction among my colleagues as educational policies have increased
emphasis on testing and databased accountability. Major changes to policy emphasize data
collection on student performance, including standardized test scores in the area of reading and
math as a required component of a teacher’s performance evaluation. This has presented unique
challenges to teachers of visual art and design beyond that of other colleagues in non-
performance-based disciplines.
During my undergraduate experience in the late 1980s, I was influenced by the Discipline
Based Art Education (DBAE) movement. When I started my career, I felt respected as an
educational professional. I was able to obtain the necessary funding for educational activities and
materials and was trusted by administrators to make curricular decisions that were in the best
interest of my students. Although the school district I taught in had a high minority population
from a low socio-economic background and most students performed poorly on standardized
tests, there were opportunities to grow the department by adding courses relevant to the
contemporary needs of students. The school was aware of its low performance status, but the
diverse needs of students were recognized by supporting robust choices of electives to help keep
students engaged. As educational policies changed, specifically with the onset of No Child Left
4
Behind (NCLB), my school was placed on the academic watch list and identified as a failing
school. I started to perceive a lack of trust in teachers and a shift away from diverse course
offerings for students to a greater emphasis on courses that reflected content in standardized
tests.
Test-based accountability was on the rise, and every department was required to create
and implement a district-wide written pre-, mid- and post-test in scantron format to collect data
on student performance; the art department was no exception. With diverse course offerings that
ranged from two-dimensional to three-dimensional and beginning to advanced levels, the only
content specific to all art and design classes that could be tested in multiple-choice form was the
elements and principles of design and vocabulary questions based on techniques specific to art
making. As a department we created district-wide tests to be compliant, although these were not
authentic and did not effectively interrogate the learning that occurred throughout the school
year. These tests were viewed as a necessary evil that we completed to demonstrate
acquiescence; however, no one seemed to pay attention to our test scores, only the assurance
they were administered and scored (Filbin & Boughton, 2016). These tests were administered
three times a year and were quite different than the normal routine of my art classes that
emphasized art making and performance-based learning. My colleagues and I were
inexperienced with evaluation and wanted to comply, not wanting to draw negative attention to
our department. Needless to say, students complained about why they had to take a multiple-
choice test in art because it did not make sense. We tried to pacify their complaints by assuring
them it was mandated three times a year and it would not negatively impact their grade. We
asked them to just get through it so we could return to our art making activities. As teachers, we
felt comfortable with our compliance; after all, the rest of the school year we were left alone. No
5
one seemed concerned with our assessment practices, and the instructional methods in our
discipline were supported. We demonstrated success in our department by participating in
multiple art shows and competitions every year, we had students who were awarded scholarships
to attend colleges to pursue careers in the arts, and we produced murals or other artwork for the
school community whenever it was requested. Generally, the teachers in the art department felt
valued. None of my evaluating administrators had any experience in the visual arts and were
usually impressed with what was happening in my classes and the artwork my students
produced. I received excellent performance evaluations and rarely was I offered suggestions on
how to improve my practice.
As NCLB continued, the school never achieved the meets or exceeds benchmarks for
Adequate Yearly Progress (AYP) and our subgroups did not make safe harbor, the minimal
required level of growth, so more restrictions were put in place. The first noticeable change in
the art department in the district was a reduction in both staff and budget. Slowly our supply
budget decreased, and if a department member left, the position was not re-filled. In addition,
there was a noticeable increase in workload with course offerings staying the same although the
number of staff members decreased. In some instances, multiple classes were run concurrently in
the same classroom at the same time to maintain full class sizes and as a way to offer upper level
courses that generally were smaller in size.
My principal needed a new teacher to lead the School Improvement Plan (SIP) committee
and asked me to be the chairperson. I wondered why he chose me over other teachers in tested
disciplines. He assured me I was a good teacher, I was well liked and respected by my
colleagues, and since I was not an English or math teacher, I was the perfect person to deliver
bad news to the faculty in a non-confrontational way. He was right; faculty attendance at SIP
6
committee was higher than ever. Meetings were filled with caring faculty members who were
willing to give up their plan time to try to find strategies to help our students. They were willing
to put in extra time to participate in subcommittees with the hope of getting us off the academic
watch list. The subcommittees were tasked with creating activities the whole school could
implement, and the SIP committee reported the results to the state. One of the plans put in place
included school-wide reading. All staff and students were asked to stop what they were doing,
put all work aside, and do silent reading for enjoyment for 20 minutes. This was closely
monitored, and it was not unusual for administrators to walk through classrooms to ensure
everyone was participating. The reading in my art room, however, was usually not very silent or
enjoyable. Most of the time was spent fielding complaints and arguments by students who were
begging to return to their art activities.
Other plans included school-wide math and school-wide writing. All faculty were given
math worksheets or writing prompts to complete with classes, and teachers were responsible for
grading and reporting student scores on a bi-weekly basis. The students complained about this
too, but I did the best I could to encourage them. Not being a certified English or math teacher, I
must admit my grading may have been a bit flawed. Although as the chairperson I understood
why this was being done, I also became a bit resentful for having to stop teaching and implement
other subject matter in my class time. Wanting to be a good teacher leader and continue to
receive excellent evaluations, I did these school-wide activities with my art classes, trying to hide
my frustration. Deeper than the frustration with the interruption to my class time, I was
becoming resentful of the policy-based disrespect to the value of my discipline. Outside of these
activities and SIP meetings, I was also privy to closed door sessions with administrators. I took
notes and helped fill out the copious amounts of forms to record test performance data and break
7
down the data by discipline and sub-group. Our Black and Hispanic sub-groups performed far
below benchmarks; however, our group of Black males struggled the most academically.
Considerable effort was given to trying to find ways to help this group of students succeed. As I
helped administrators with the endless documentation of what our school was doing to help
students achieve, it became evident to me there was no way we could raise scores enough to ever
get off the watch list. I also began to understand that a lot of decisions that affected students were
out of the hands of administrators and teachers who dealt with students every day and in the
hands of policymakers who knew nothing about teaching. As the achievement benchmarks
continued to rise, our achievement gap widened. I knew it was time for me to step down from
this leadership position; I could clearly see there was no place for the arts according to these
policies and my time would be better spent on my students in my classroom.
When the movement to College Readiness Standards (CRS) began, there was some hope
that we could demonstrate the good things our district had to offer through these standards.
When my school sought to increase our AP program, the art department obliged by adding AP
Studio Art, yet no additional art teachers were hired to assist in handling this workload unlike
other departments in the school that saw an increase in staff members. When we advocated for
more teachers, we were told there was nothing in the budget for the arts because they were not
part of the core curriculum. Additional funding was to be allocated for reading and math to help
raise test scores. Accountability requirements continued to increase, and school-wide
professional development on validity and reliability on our teacher generated tests was offered to
all departments to help with data analysis on our mandated pre-, mid- and post-tests. After this
professional development, the art department had a data meeting with our department
administrator who had no background in the arts. We were reprimanded because our students
8
performed too well on pre-tests and we would not be able to show adequate growth throughout
the school year. Students who scored 100% on their pre-test were a detriment to our growth data
because they could not show any growth. Under the new student growth policies, we were
considered failures as teachers. I became extremely frustrated with this ludicrous comment and
blurted out in the meeting, “If we have high schoolers that don’t know what a line or a shape is, I
think we have bigger problems.” As a department we explained to our administrator that the
language of art and identification of techniques were where our classes began and had little to do
with measuring what our students actually learned in our classes or the final outcomes of our
courses. Our curriculum was concerned with what the students could do by applying this
knowledge to actual art making. The consistently high pre-test scores across all art classes
demonstrated reliability; however, we quickly realized the scores were not valid because they
had nothing to do with what our students were actually learning in our classes (Filbin &
Boughton, 2016). By using the model of portfolio assessment that was introduced to us when we
added AP Studio Art, we were able to present evidence to our administration and the district-
wide curriculum committee that the multiple-choice tests we were administering in the
department were not valid. We were able to provide an alternate assessment format the art
department could implement that utilized only performance-based assessments scored with
rubrics and a student portfolio in lieu of a written right/wrong final exam for all art classes, thus
eliminating our multiple-choice tests.
Because our district offered AP Studio Art courses and the AP student exams were
portfolio based, we were able to advocate for portfolio assessments to be implemented
department-wide in lieu of any right/wrong types of tests. In reflection, what seemed like
winning the war was only a small victory. I realized the problem my art department faced was a
9
symptom of a much deeper systemic problem in American public schools. It was quite clear to
me the importance of assessment practice in visual art and design education, and the inherent
flaws in educational policies that were shaping the teaching profession. I sought answers to
questions that no one in my school could provide. I made the decision to pursue my doctorate to
find what assessment practices were best for the field so I could help other art educators who
might be struggling with similar questions.
Filbin, D. N. & Boughton, D. G. (2016). Educational policy and mentorship: Transforming
classroom practice and assessment in the art class. Visual Inquiry Learning and Teaching
Art. 5(3), 263-272.
