ArticlePDF Available

Abstract and Figures

Student evaluation of teaching (SET) has become a ubiquitous feature of higher education. The attainment and maintenance of positive SET is essential for most teaching staff to obtain and maintain tenure. It is not uncommon for teachers to receive offensive and non-constructive commentary unrelated to teaching quality. Regular exposure to SET contributes to stress and adversely impacts mental health and well-being. We surveyed Australian teaching academics in 2021, and in this paper, we explore the perceived impacts of SET on the teaching and learning experience, academic standards and quality. Many respondents perceived that SET contributes to an erosion of standards and inflation of grades. A thematic analysis of open-ended questions revealed potential mechanisms for these impacts. These include enabling a culture of incivility, elevating stress and anxiety in teaching staff, and pressure to change approaches to teaching and assessment to achieve the highest scores. Playing the SET game involves balancing a commitment to quality and standards with concessions to ensure optimal student satisfaction. Anonymous SET is overvalued, erodes standards and contributes to incivility. The process of SET needs urgent reform.
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=caeh20
Assessment & Evaluation in Higher Education
ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/caeh20
Playing the SET game: how teachers view the
impact of student evaluation on the experience of
teaching and learning
Richard Lakeman, Rosanne Coutts, Marie Hutchinson, Debbie Massey, Dima
Nasrawi, Jann Fielden & Megan Lee
To cite this article: Richard Lakeman, Rosanne Coutts, Marie Hutchinson, Debbie Massey, Dima
Nasrawi, Jann Fielden & Megan Lee (2022): Playing the SET game: how teachers view the impact
of student evaluation on the experience of teaching and learning, Assessment & Evaluation in
Higher Education, DOI: 10.1080/02602938.2022.2126430
To link to this article: https://doi.org/10.1080/02602938.2022.2126430
Published online: 25 Sep 2022.
Submit your article to this journal
View related articles
View Crossmark data
ASSESSMENT & EVALUATION IN HIGHER EDUCATION
Playing the SET game: how teachers view the impact of
student evaluation on the experience of teaching and
learning
Richard Lakemana , Rosanne Couttsa , Marie Hutchinsona , Debbie
Masseya , Dima Nasrawia , Jann Fieldena and Megan Leea,b
aFaculty of Health, Southern Cross University, Lismore, Australia; bFaculty of Society & Design, Bond University,
Australia
ABSTRACT
Student evaluation of teaching (SET) has become a ubiquitous feature
of higher education. The attainment and maintenance of positive SET is
essential for most teaching staff to obtain and maintain tenure. It is not
uncommon for teachers to receive offensive and non-constructive com-
mentary unrelated to teaching quality. Regular exposure to SET contrib-
utes to stress and adversely impacts mental health and well-being. We
surveyed Australian teaching academics in 2021, and in this paper, we
explore the perceived impacts of SET on the teaching and learning
experience, academic standards and quality. Many respondents perceived
that SET contributes to an erosion of standards and inflation of grades.
A thematic analysis of open-ended questions revealed potential mech-
anisms for these impacts. These include enabling a culture of incivility,
elevating stress and anxiety in teaching staff, and pressure to change
approaches to teaching and assessment to achieve the highest scores.
Playing the SET game involves balancing a commitment to quality and
standards with concessions to ensure optimal student satisfaction.
Anonymous SET is overvalued, erodes standards and contributes to
incivility. The process of SET needs urgent reform.
In most universities, students are routinely surveyed regarding their teaching and learning
experiences. Anonymous internet-based surveys are usually conducted at the end of teaching
terms. These invite the rating of teachers and the learning experience on Likert rating scales
and provide opportunities for students to comment on their teachers and courses. The assumed
and often stated purpose of student evaluations of teaching (SET) is to provide a means to
document and improve teaching quality (Hammonds et al. 2017). However, despite decades of
use, there is a body of evidence and opinion suggesting that SET has drifted far from that
purpose (Stroebe 2020). SET is now conceptualised as enabling students to have a ‘voice’ (Stein
et al. 2021). SET also enables rankings and comparison of teachers, courses and universities
based on student opinion and satisfaction (Harvey 2022). Some researchers have questioned
the validity and reliability of SET and asserted that unmoderated student opinion should not
influence teacher career advancement or job security (Bedggood and Pollard 1999). However,
SET does impact teaching and learning experiences in profound ways. Maintaining positive SET
has become pivotal to the ongoing tenure and promotion of academic staff (Hornstein and
Law 2017).
© 2022 Informa UK Limited, trading as Taylor & Francis Group
CONTACT Richard Lakeman richard.lakeman@scu.edu.au
https://doi.org/10.1080/02602938.2022.2126430
KEYWORDS
Student evaluation of
teaching; student
feedback surveys;
teaching and learning;
academics
2 R. LAKEMAN ETAL.
Anonymous SET is now a ubiquitous feature of the neoliberal university governed by prin-
ciples of new managerialism (Shepherd 2018). Managerialism and the practices derived from
this ideology have been found to contribute to the occupational stress of academics (Lee, Coutts
et al. 2022). Anonymised and standardised SET processes enable managers to oversee the
teaching encounter, foster competition between academics, and position the student as a cus-
tomer and education as a commodity (Shepherd 2018). The massification of SET has been
accompanied by the entrenchment of managerialism and profound changes in the relationship
of teaching staff with the university and with students.
SET, as originally conceived and practised for most of the twentieth century, was a private
matter between teachers and students (Algozzine et al. 2004). Anonymity in such surveys was
assumed partly because of the power differential between tenured faculty and students. There
is evidence that anonymity leads to more critical feedback when medical students are asked
to rate supervising physicians (Afonso et al. 2005). However, the majority of teaching staff today
do not have any influence over the career trajectories of students and often have a tenuous
relationship with the universities in which they teach.
In Australian universities, since 1989 (as illustrated in Table 1), there has been a 368% increase
in the number of students enrolled and more than a doubling of the ratio of students to full-time
equivalent teaching staff (excluding research only and other roles) to one teacher to 35.7 students
in 2021 (Department of Education 2022). The growth in teaching staff was only 40% over 30 years
to 2020. Following a massive shedding of teaching staff in response to COVID-19 in 2021 this
represented a growth of only 14% over 30 years. Well above 60% of the teaching workforce were
estimated to be casually employed prior to COVID-19 (Evans et al. 2019). Increasing casual and
precarious employment in higher education has been noted internationally as a trend and con-
cern (Mason and Megoran 2021; Leathwood and Read 2022). Casual or sessional teaching staff
have little power over students or to effect change in teaching or learning practices. Yet, this is
the group most likely to be rated in routine SET and suffer the consequences.
Most SET has a poor correlation with student learning and is often biased against women
and other marginalised groups (Kreitzer and Sweet-Cushman 2022). A recent review found
that SET is heavily influenced by factors that have little to do with course quality and effec-
tiveness (Heffernan 2022). SET is increasingly being used as a vehicle to make sexist, racist,
prejudiced and abusive comments towards marginalised groups (Heffernan 2022). SET is
influenced by the personal characteristics of the instructor, as well as the individual biases
of students (Wallace, Lewis, and Allen 2019). Whilst the personal characteristics of teaching
staff cannot be changed, elements of the teaching encounter, such as offering inducements,
can bias SET scores favourably. For example, Hessler et al. (2018) distributed cookies to a
group of medical students prior to the administration of SET, and this led to significantly
higher SET scores compared to a cookie-deprived control group. It is widely acknowledged
that SET scores are skewed by unhappy students (Clayson and Haley 2011; Uttl and Smibert
2017; Wallace, Lewis, and Allen 2019; LeFebvre, Carmack, and Pederson 2020; Heffernan 2022;
Lakeman et al. 2022).
Table 1. Student enrolments and teaching ratios in Australian universities.
Year: 1989 2000 2010 2020 2021
Number of students commencing 181092 285518 487917 641235
Number of students continuing 259982 409966 704740 981632
Total number of enrolled students 441074 695484 1192657 1622868 ?
Total number of teaching academics 26895 30725 37918 45004 31196
Ratio of teaching staff to students 1:16.4 1:22.6 1:31.5 1:35.7
Total number employed in administrative / support roles 33045 41732 56327 71641 61218
Ratio of administrative/support roles to students 1:13.3 1:16.6 1: 21.2 1:22.7
Ratio of teaching staff to administrative/support roles 1:1.2 1:1.4 1:15 1:1.6 1:2
Source: Department of Education (2022). Note that teaching and administrative roles are reported as full-time
equivalent.
ASSESSMENT & EVALUATION IN HIGHER EDUCATION 3
Stroebe (2020) has asserted that SET encourages grade inflation, rewards poor teaching, and
punishes those who grade strictly or instruct in challenging courses. Carpenter, Witherby, and
Tauber (2020) have found that student evaluation of teaching effectiveness can be a poor
predictor of actual learning. They note that students intuitively but incorrectly believe that they
learn more when instructors are engaging, have well-polished lectures and don’t require active
participation from students. It is possible that over-reliance on positive and well-meaning sug-
gestions for improvement derived from SET might alter the teaching and learning environment
in ways that do not enhance learning.
Given the extent of change in the higher education sector, and the plethora of stressors
facing teaching staff in higher education, it remains unclear what the relative impact of SET
has on the experience of teaching and learning or notions of quality. This study aimed to
explore the perceptions of teaching staff on how anonymous SET has impacted their experience,
professional relationships, and teaching and learning quality.
Methods
This paper reports the findings of quantitative and qualitative survey responses, extending on
previous analysis by Lakeman et al. (2021, 2022). These previous publications elicited many
examples of anonymous comments associated with SET, which were insulting, projected blame,
addressed in an insulting way people’s appearance and attire, made unfounded allegations, or
were threatening and punishing (Lakeman et al. 2021). Further examination of the perceived
impact of potential or actual regular exposure to defamatory, hateful or insulting comments
on people’s health suggested that the impacts are sometimes severe (Lakeman et al. 2022).
These included but were not limited to ongoing or anticipatory stress, post-traumatic stress
responses, chronic health problems, and coping strategies such as excessive alcohol consumption
with health impacts (Lakeman et al. 2022).
Procedure
This was a descriptive study involving quantitative descriptive analysis of survey responses and
a thematic analysis of free-text comments included alongside these questions. An anonymous
self-report survey was conducted using snowball sampling on social media platforms Facebook
and Twitter and distributed to staff emails of Australian universities. A link to the survey in The
Conversation (Lee, Nasrawi et al. 2021) increased the number of participants who completed
each item on the survey. The survey was comprised of 30 questions. This paper reports on the
10 questions related to impacts on teaching quality and the teaching and learning experience.
Respondents were presented with questions requiring responses on Likert scales (see Table 2)
followed by related open questions. The data obtained from the open questions were combined
in the thematic analysis, as they often addressed similar issues or respondents made reference
to previous responses. The questions asked were:
• Overall, how has anonymous narrative feedback impacted your teaching or academic
practice?
• Please comment or elaborate on how you perceive anonymised narrative feedback has
impacted the teaching and learning experience
• Please comment or elaborate on how you perceive anonymous narrative feedback has
impacted your personal and professional relationships.
The analysis of the remaining 12 questions are reported in other publications (Lakeman et al.
2021, 2022). Ethical approval for this study was obtained from the Human Research Ethics
Committee at Southern Cross University (2021/047).
4 R. LAKEMAN ETAL.
Table 2. Frequency and percentages of anonymous narrative student feedback’s impact on teaching
and learning.
Never Sometimes
About half
the time
Most of the
time Always
Statement n%n%n%n%n%Total
Narrative student feedback accurately
reflects teaching skill, ability, or
competence.
35 5.0% 278 39.9% 139 19.9% 235 33.7% 10 1.4% 697
Anonymous narrative student feedback
is related to teaching quality.
66 9.5% 342 49.1% 140 20.1% 140 20.1% 91.3% 697
Anonymous narrative student feedback
has undermined my professional
confidence
78 11.3% 124 17.9% 267 38.5% 145 20.9% 79 11.4% 693
Anonymous narrative student feedback
can lead to a lowering of academic
standards
118 16.9% 357 51.2% 75 10.8% 116 16.6% 31 4.4% 697
Anonymous narrative student feedback
can lead to inflation of student
grades
282 40.5% 259 37.2% 44 6.3% 86 12.3% 26 3.7% 697
Anonymous narrative student feedback
has negatively impacted my
relationships with colleagues
288 41.8% 188 27.3% 171 24.8% 34 4.9% 81.2% 689
Anonymous narrative student feedback
has negatively impacted my
relationship with students
191 27.7% 175 25.4% 240 34.8% 63 9.1% 20 2.9% 689
Data analysis
Descriptive statistics were examined using percentages and frequencies of categorical data and
means, standard deviations and histograms for continuous data. The responses to the free-text
responses were collated and analysed inductively to identify the major themes. The process was
informed by DeSantis and Ugarriza’s (2000) description of procedures for consensus coding in
which four researchers simultaneously undertook a content analysis and then categorised the
responses jointly. Consensus coding can enhance the trustworthiness of the findings (Armat
et al. 2018).
Results
The data from 697 respondents who completed most questions outlined in Table 2 and responded
to at least one open question were included in this analysis. This sample was predominantly
female (n = 535), with a mean age of 48.7 years (SD = 10). 429 (61%) had continuing and per-
manent employment, and 268 (38.5%) were employed casually (n = 97) on fixed terms (n = 85)
or were on probation (n = 86). Respondents reported a mean of 13.6 years of teaching (SD =
8.2) and 58% of their role involving teaching (SD = 27%).
The aggregated ratings of questions relating to teaching and learning quality are outlined
in Table 2. These questions addressed opinions about the extent to which anonymous SET
impacted the teaching and learning experience and issues relating to academic standards and
quality. The majority (>90%) perceived that anonymous SET accurately reflected teaching skills,
ability or competence and was related to teaching quality at least some of the time. However,
a majority (>80%) also perceived that SET had undermined their professional competence and
could lead to a lowering of academic standards. It is noteworthy that 40.5% perceived that
anonymous SET never led to grade inflation. Over 70% reported that SET negatively impacted
relationships with students at least sometimes, whilst 58% reported that professional relation-
ships were sometimes negatively impacted.
Responses to open questions were largely critical and elaborated on interpersonal processes,
dynamics (between students, colleagues and faculty) and teaching and learning processes and
ASSESSMENT & EVALUATION IN HIGHER EDUCATION 5
outcomes perceived to be impacted by SET. Responses clustered under four related themes: (i)
enabling a culture of incivility, (ii) elevated stress (iii) erosion of standards and quality, and (iv)
playing the SET game.
Enabling a culture of incivility
The majority of respondents perceived that anonymous SET impacted negatively on relationships
with colleagues and students. SET was perceived to both contribute to and reflect a culture of
incivility whereby values such as collegiality and genuine mutual respect were eroded. Most
respondents were aware that non-constructive, highly personalised and offensive comments
entirely unrelated to teaching are invalid and exemplify incivility. However, uncritical comments
and high ratings were often perceived to be valid, illustrative of excellence and indicative of
higher relative standing with peers. Some expressed resentment that they or peers would be
congratulated for perceived achievements that had little to do with excellence in teaching or
even good practice. One respondent noted that SET causes:
...friction with colleagues due to misinterpretation of feedback… [and leads to] being targeted by students…
being seen as a soft touch… and seeking pathetic validation and assurance from others.
Others reported making negative evaluations of themselves and colleagues for being drawn
into competition for positive evaluations and feedback. Some reported feeling uncomfortable
with their response of sympathy, relief or pride when others received negative comments or
ratings. Some reported losing respect for colleagues whom they perceived deliberatively
attempted to manipulate their relative standing with students:
Staff start to compete and show resentment towards others who have higher scores. A colleague once talked
to students behind my back, trying to lower my score… because I get high scores. These surveys bring out the
worst in people.
A majority of respondents reported that SET undermined their professional confidence at times.
Many reported thinking that they were ‘bad teachers’, and at times they were embarrassed by
their scores or comments. Some reported being acutely fearful that they would lose their job
because of poor feedback. SET was seen as facilitating and encouraging comparison, competition
and the amplification of uncivil behaviour and poor relationships:
[SET leads to]… a culture of bragging about high results, and competition between colleagues… [SET] affected
my relationship with another colleague… I was accused of pandering to students and being too available for them.
It was widely acknowledged and understood that students utilise SET to bully, wound and
attempt to inflict harm on teachers at times. However, for many, it was reported to be more
demoralising that SET was used for these purposes by managers and colleagues:
Feedback [is] adopted by colleagues as a tool to bully and target other academics.
Anonymised feedback was described as ‘sucking the joy’ out of teaching. These sentiments
were more common in those who had commenced teaching before the massification of anon-
ymous SET in universities. Whilst many people (41.8%) did report that SET did not negatively
impact collegial relationships with peers, some noted that it was difficult to mobilise support
around the issue of negative SET as it was frequently a consideration in performance reviews.
Drawing attention to personalised feedback or low relative scores could be potentially dangerous
and overly defensive behaviour taken as an indicator that the feedback has some validity:
My passion for teaching decreased due to negative comments… [there is]… now no social courtesy… [I have]
difficulty in finding joy in teaching… I have a hesitancy to talk to supervisors… so that they don’t think I’m
making excuses.
6 R. LAKEMAN ETAL.
Elevated stress
Participants in this study highlighted the impact of anonymous student feedback on their
physical and psychological well-being. SET contributed directly and indirectly to occupational
stress and associated health impacts. These health impacts included vomiting, sleeplessness,
headaches, stomach cramps, skin rashes, elevated blood pressure as well as anxiety, stress and
medically diagnosed burnout. For example:
Sleeplessness, anxiety, anger, loss of self-confidence, loneliness, stress, imposter syndrome, depression, suicide,
internalised feelings, vulnerability, frustration, trauma, short temper, irritability, burnout, general ill health, feeling
of shame, dejection and disappointment…
The stressors which were particular to SET included the sense of being under continuous
surveillance and being judged by peers and students. It was perceived to be particularly
egregious to have no right to reply when vexatious comments or satisfaction scores were
negatively skewed by outlier students. The epistemic injustice associated with having to defend
oneself against false or vexatious claims engendered weariness and heightened anxiety in
dealings with students:
...feeling hopeless as I cannot reply to this negative feedback… resentment towards students… poor engagement
in class… [I’m] very careful about what I say in front of my students…[teaching is] tinged with fear, worry, and
paranoia.
The personal impacts of perceived surveillance in the workplace and increased stress led
many respondents to report needing to debrief at home or being unable to ‘switch off’ from work:
[SET has] affected my personal/home life (over a short period), [it is] overwhelming and I withdraw… I need to
debrief at home… my partner is sick of hearing about evaluation time.
So embedded is the expectation that negative and personally wounding comments would
occur that some people reported feelings of guilt for their emotional responses. Some spoke
of the need to manage the stress better and to personally ‘toughen up’:
Learning to manage feedback is important… [dealing with], feelings of guilt for not being able to manage
feedback, to toughen up and develop a thick skin. A better way is required to help us manage feedback at work
and not take it home.
Erosion of standards and quality through manipulation of the teaching encounter
Respondents described how students are now perceived as consumers rather than learners in
the higher education sector. Academics were clear that long-held educational values were being
overshadowed by a range of managerial imperatives such as high measures of consumer sat-
isfaction, retention and success. Whilst not everyone agreed, most respondents perceived SET
contributed to an erosion of academic standards:
...it was common knowledge and discussion amongst staff that SET has led to lowering of academic
standards.
Many respondents described how they felt they were being ‘manipulated’ by students and
administrators. In order to placate students and achieve high satisfaction scores, they felt under
pressure to change their approach to teaching the subject material:
I have seen many academics butcher their own standards to get good feedback to help with promotions and
alleviate line manager actions.
The university management takes students’ comments as a KPI [key performance indicator] and puts pressure
and stress on academics to achieve good comments from students.
ASSESSMENT & EVALUATION IN HIGHER EDUCATION 7
Various strategies, including the inflation of student grades and adjustments that might
support ‘being popular’, were seen as necessary accommodations in teaching life to avoid or
reduce damaging feedback. Such accommodations ensured one’s performance was not called
into question:
Anonymous student feedback is used at my university in performance reviews, and I am aware that some aca-
demics are adjusting their students’ learning experiences and grades to attain positive feedback.
Student evaluations were described as a ‘managerial weapon’, influencing tenure, employment
and promotion opportunities and routinely used in the ‘management of performance’. Some
respondents stated that the changes to teaching and learning that they were undertaking had
become a ‘self-preservation’ strategy:
There was incredible pressure on me to lower academic standards and avoid failing students in order to avoid negative
feedback. While I resisted this pressure, this did lead to a situation where I was directly cautioned by the program
director that my employment would not continue unless the feedback for the course in the following year improved.
Respondents describe working in a ‘culture of fear’ with the outcome being that students
were perceived by some to be able to obtain a qualification regardless of their ability or effort:
It has resulted now in the harsh reality that provided you are enrolled, you are going to get a degree. It doesn’t
matter whether the student has tried or not. It has resulted in a culture of fear whereby academics are so terrified
of receiving negative feedback… that marking and teaching standards are so low that people are getting degrees
who shouldn’t.
Playing the SET game
Student feedback was likened to a ‘popularity contest’, where the easiest response was to ‘play
the SET game’ by making changes to the curriculum in order to placate and even ‘please’ the
students:
...fear of negative feedback causes some academic staff to grade students’ work more highly than it deserves.
Most of the time I see anonymised narrative feedback as feedback about the lecturer’s popularity and it mostly
has little to do with the quality of learning or teaching.
Respondents describe strategies such as inflating grades, ‘dumbing down’ curricula, and
reducing student workload expectations. There was also an acceptance by many that standards
and teaching quality were eroding:
Quality standards are dropping. Content and rigorous assessment are penalised… students complain, and the
next iteration will have content weakened and assessment reduced in thoroughness.
Many respondents acknowledge strategically making what they describe as unnecessary
concessions or accommodations to attempt to secure good ratings. Others described resisting
engaging in the SET game but still being penalised:
I believe that the anonymous narrative student feedback can lead to reducing academic rigour, I have held on
to the high quality and standards that are expected in my units and believe I am penalised by student
feedback.
The lowering of standards was lamented by many. A number of respondents representing
health disciplines were aware of the impact of SET on the education of ‘safe’ and ‘competent’
practitioners. One respondent noted that failing a student in the interest of protecting the
public might ultimately lead to negative consequences for the academic:
If I fail a student, it is because they have not met the required standards. Clinical safety and competence are
absolutely paramount. The failing student can give negative and upsetting reviews because it is anonymous,
which impacts on the academic’s progression.
8 R. LAKEMAN ETAL.
Others expressed the view that it is easier to lower standards than being criticised by stu-
dents, and keeping students’ happy’ is a way to avoid negative feedback. This balance of
maintaining standards and keeping the customer happy is the essence of the SET game.
Discussion
A limitation of this research is that it reflects the opinions of a sample of Australian academics
after COVID-19 austerity measures had impacted the higher education sector. Thus, many ten-
uously employed or ‘stood down’ academics who may have been most affected by SET would
not have had the opportunity to participate. However, that most people reported moderate
ratings on the Likert scale questions and a substantial minority perceived that SET did not
contribute to grade inflation adds credibility to the findings. This was not a sample of people
with extreme views. The sample consisted of experienced and credible teaching staff in Australia
who were well placed and qualified to tease out the impacts of SET on teaching quality against
a background of rapid change in the sector.
This study has highlighted how SET can and is perceived to impact the teaching and learning
experience of academics and students. Participants indicated that the anonymous nature of the
SET promotes a culture of incivility both among students and staff. This incivility engenders resent-
ment and competition rather than collaboration and cohesion. SET negatively impacts relationships
between academic staff who compete for recruitment, promotion and funds (Hessler et al. 2018).
This relentless competition can contribute to burnout, impacting the recruitment and retention of
staff and the student experience (Singh et al. 2020). We argue that by ‘playing the SET game’, faculty
contribute to the erosion of teaching quality and standards. The manipulation of content and
process continues despite the ongoing acknowledgment that SET is impractical, lacks validity and
is primarily a measure of customer satisfaction (Spooren, Brockx, and Mortelmans 2013).
SET continues to be promoted and over-valued by universities. The respondents in our study
spoke of lowering teaching standards and inflation of grades to obtain positive SET scores in
order to satisfy management and secure tenure and promotion. These informed opinions suggest
this is now an entrenched part of academic culture. Administrators and managers need to
consider the potential costs (in terms of quality and standards) of attaining or aspiring to
exemplary ratings of satisfaction and success. As Carpenter, Witherby, and Tauber (2020) note,
great care should be taken in interpreting comments or recommendations in SET as students
do not always have insight into what is helpful for their learning at the time. Our findings are
in accord, and further suggest that the position of SET as a primary factor in performance
reviews and promotion has contributed to a lowering of standards and a culture of incivility.
The anonymity and timing of SET at times of heightened anxiety for students also amplify incivility.
Unsurprisingly, some anxious students project blame on teachers for their own potential failings and
exploit the opportunity of anonymity to wound others. This can have serious health consequences
(Lakeman et al. 2022). In response to the high prevalence of invalid, uncivil and defamatory anony-
mous comments in SET (Lakeman et al. 2021), computer models have been developed to screen and
remove abusive or harmful comments in routine SET (Cunningham et al. 2022). However, such screen-
ing and unveiling of the identity of perpetrators who breach a faculty code of conduct does not
necessarily improve the validity of SET. Scores can still be skewed by dissatisfied students motivated
to punish teachers, regardless of their politely expressed comments. It is timely to unshackle SET
entirely from anonymity, which as Brown (2015, 170) notes in itself, could be transformative:
Dare greatly and put your name on your posted comments online. If you don’t feel comfortable owning it, then don’t say
it. And if you’re reading this and you have control over online sites that allow comments, then you should dare greatly
and make users sign in and use real names, and hold the community responsible for creating a respectful environment.
Some higher education institutions have reported greatly enhancing the quality of their
educational offerings, and enhanced success and satisfaction through solely eliciting identified
and qualitative feedback from students (Bartkowiak-Théron, McShane, and Knight 2020)
ASSESSMENT & EVALUATION IN HIGHER EDUCATION 9
In our study, respondents reported that SET was helpful at times in improving teaching.
However, the over-privileging and reliance on SET as an indicator of teaching quality creates a
climate of competition and distrust where inappropriate outcomes are valued (Antoci et al. 2021).
Defining quality teaching is challenging because it is a multifaceted, complex construct that is
hard to evaluate (Crebbin 1997). Thus, expecting students to be the primary arbiters of teaching
quality is fraught with problems. Students may need coaching on delivering constructive feedback
and ensuring their collective voices are used to improve teaching quality rather than dilute it.
Conclusion
SET is a poor measure of student learning, teaching quality and teacher performance. Embedded
within routine administrative systems, anonymised SET has become an administrative tool that exerts
managerial surveillance and control over academics. When used to scrutinise the teacher, rather than
meaningfully evaluate teaching and learning, SET has the potential for misuse. Our findings highlight
the invidious situation higher education teachers find themselves in, playing the SET game to survive
the toxic environment. What is absent from the system is any clear definition of what constitutes
‘quality’ or ‘excellence’. Objectified and publicly scrutinised, teaching staff monitoring their own
behaviour internalise responsibility for the relentless cycle of anonymous and often personalised
commentary. This rationality also shapes students by conscripting them into taking part in divisive
and objectifying practices. If SET is to inform teaching quality and educational practice, it needs to
be collected ethically. SET processes need to be constructive, collaborative and respectful, not
over-valued or used as a weapon to wound or malign teachers in higher education.
Disclosure statement
No potential conflict of interest was reported by the authors.
Notes on contributors
Richard Lakeman is an Associate Professor at Southern Cross University and Coordinator of SCU Online Mental
Health Programmes. He is a Mental Health Nurse, psychotherapist and fellow of the Australian College of Mental
Health Nursing.
Rosanne Coutts is an Associate Professor in the Faculty of Health at Southern Cross University and teaches in
the field of psychology.
Marie Hutchinson is a Professor in Nursing at the Faculty of Health, Southern Cross University.
Deb Massey is an Associate Professor at Southern Cross University and an intensive care nurse.
Dima Nasrawi is Lecturer in Nursing at Southern Cross University and a PhD student at Griffith University. She
is a Cardiac Nurse and a member of the Australian Cardiac Rehabilitation Association.
Jann Fielden is a Casual Lecturer at Southern Cross University.
Megan Lee is a Senior Teaching Fellow in Psychology at Bond University and an Adjunct Senior Lecturer in
Education at Southern Cross University. Megan is a member of the American Psychological Association (APA) and
the Australasian Psychological Society (APS).
ORCID
Richard Lakeman http://orcid.org/0000-0002-4304-5431
Rosanne Coutts http://orcid.org/0000-0001-5064-8903
Marie Hutchinson http://orcid.org/0000-0003-0025-2681
Debbie Massey http://orcid.org/0000-0002-0466-1960
Dima Nasrawi http://orcid.org/0000-0002-2878-2468
Jann Fielden http://orcid.org/0000-0003-0084-0838
Megan Lee http://orcid.org/0000-0001-7531-5312
10 R. LAKEMAN ETAL.
References
Afonso, N. M., L. J. Cardozo, O. A. J. Mascarenhas, A. N. F. Aranha, and C. Shah. 2005. “Are Anonymous
Evaluations a Better Assessment of Faculty Teaching Performance? A Comparative Analysis of Open
and Anonymous Evaluation Processes.Family Medicine 37 (1): 43–47.
Algozzine, B., J. Gretes, C. Flowers, L. Howley, J. Beattie, F. Spooner, G. Mohanty, and M. Bray. 2004.
“Student Evaluation of College Teaching: A Practice in Search of Principles.College Teaching 52
(4): 134–141. doi:10.3200/CTCH.52.4.134-141.
Antoci, A., I. Brunetti, P. Sacco, and M. Sodini. 2021. “Student Evaluation of Teaching, Social Influence
Dynamics, and Teachers’ Choices: An Evolutionary Model.” Journal of Evolutionary Economics 31 (1):
325–348. doi:10.1007/s00191-020-00705-4.
Armat, M. R., A. Assarroudi, M. Rad, H. Sharifi, and A. Heydari. 2018. “Inductive and Deductive:
Ambiguous Labels in Qualitative Content Analysis.The Qualitative Report 23 (1): 219–221.
doi:10.46743/2160-3715/2018.2872.
Bartkowiak-Théron, I., A. L. J. McShane, and M. G. Knight. 2020. “Departing from Anonymous and
Quantitative Student Feedback: Fostering Learning and Teaching Development through Student
Evaluations. Journal of Applied Learning & Teaching 3 (1): 118–128. doi:10.37074/jalt.2020.3.s1.16.
Bedggood, R. E., and R. J. Pollard. 1999. “Uses and Misuses of Student Opinion Surveys in Eight Australian
Universities. Australian Journal of Education 43 (2): 129–141. doi:10.1177/000494419904300203.
Brown, B. 2015. Daring Greatly: How the Courage to Be Vulnerable Transforms the Way we Live, Love,
Parent and Lead. UK: Penguin Life.
Carpenter, S. K., A. E. Witherby, and S. K. Tauber. 2020. “On Students’ (Mis) Judgments of Learning
and Teaching Effectiveness.Journal of Applied Research in Memory and Cognition 9 (2): 137–151.
doi:10.1016/j.jarmac.2019.12.009.
Clayson, D. E., and D. A. Haley. 2011. “Are Students Telling Us the Truth? A Critical Look at the Student
Evaluation of Teaching.Marketing Education Review 21 (2): 101–112. doi:10.2753/MER1052-8008210201.
Crebbin, W. 1997. “Defining Quality Teaching in Higher Education: An Australian Perspective.Teaching
in Higher Education 2 (1): 21–32. doi:10.1080/1356251970020102.
Cunningham, S., M. Laundon, A. Cathcart, M. A. Bashar, and R. Nayak. 2022. “First, Do No Harm:
Automated Detection of Abusive Comments in Student Evaluation of Teaching Surveys.” Assessment
& Evaluation in Higher Education. doi:10.1080/02602938.2022.2081668.
Department of Education. 2022. “Higher Education Statistics.” Australian Government. Accessed 10
September 2022. https://www.education.gov.au/higher-education-statistics.
DeSantis, L., and D. N. Ugarriza. 2000. “The Concept of Theme as Used in Qualitative Nursing Research.
Western Journal of Nursing Research 22 (3): 351–372. doi:10.1177/019394590002200308.
Evans, J., S. Ruane, H. Southall, N. Dados, J. Goodman, and K. Yasukawa. 2019. “Counting the Uncounted:
Contestations over Casualisation Data in Australian Universities.” In Data in Society: Challenging
Statistics in an Age of Globalisation, edited by J. Evans, S. Ruane, and S. Humphrey, 327–336. Bristol:
Policy Press.
Hammonds, F., G. J. Mariano, G. Ammons, and S. Chambers. 2017. “Student Evaluations of Teaching:
Improving Teaching Quality in Higher Education.” Perspectives: Policy and Practice in Higher Education
21 (1): 26–33. doi:10.1080/13603108.2016.1227388.
Harvey, L. 2022. “Back to Basics for Student Satisfaction: Improving Learning Rather than Constructing
Fatuous Rankings. Quality in Higher Education. doi:10.1080/13538322.2022.2050477.
Heffernan, T. 2022. “Sexism, Racism, Prejudice, and Bias: A Literature Review and Synthesis of Research
Surrounding Student Evaluations of Courses and Teaching. Assessment & Evaluation in Higher
Education 47 (1): 144–154. doi:10.1080/02602938.2021.1888075.
Hessler, M., D. M. Popping, H. Hollstein, H. Ohlenburg, P. H. Arnemann, C. Massoth, L. M. Seidel, A.
Zarbock, and M. Wenk. 2018. “Availability of Cookies during an Academic Course Session Affects
Evaluation of Teaching.Medical Education 52 (10): 1064–1072. 10.1111/medu.13627.
Hornstein, H. A., and H. F. E. Law. 2017. “Student Evaluations of Teaching Are an Inadequate Assessment
Tool for Evaluating Faculty Performance. Cogent Education 4 (1): 1304016. doi:10.1080/233118
6X.2017.1304016.
Kreitzer, R. J., and J. Sweet-Cushman. 2022. “Evaluating Student Evaluations of Teaching: A Review of
Measurement and Equity Bias in SETs and Recommendations for Ethical Reform.Journal of Academic
Ethics 20 (1): 73–84. doi:10.1007/s10805-021-09400-w.
ASSESSMENT & EVALUATION IN HIGHER EDUCATION 11
Lakeman, R., R. Coutts, M. Hutchinson, M. Lee, D. Massey, D. Nasrawi, and J. Fielden. 2021. “Appearance,
Insults, Allegations, Blame and Threats: An Analysis of Anonymous Non-Constructive Student
Evaluation of Teaching in Australia. Assessment & Evaluation in Higher Education. doi:10.1080/0260
2938.2021.2012643.
Lakeman, R., R. Coutts, M. Hutchinson, D. Massey, D. Nasrawi, J. Fielden, and M. Lee. 2022. “Stress,
Distress, Disorder and Coping: The Impact of Anonymous Student Evaluation of Teaching on the
Health of Higher Education Teachers.Assessment & Evaluation in Higher Education. doi:10.1080/02
602938.2022.2060936.
Leathwood, C., and B. Read. 2022. “Short-Term, Short-Changed? A Temporal Perspective on the
Implications of Academic Casualisation for Teaching in Higher Education.Teaching in Higher
Education 27 (6): 756–771. doi:10.1080/13562517.2020.1742681.
Lee, M., R. Coutts, J. Fielden, M. Hutchinson, R. Lakeman, B. Mathisen, D. Nasrawi, and N. Phillips.
2022. “Occupational Stress in University Academics in Australia and New Zealand. Journal of Higher
Education Policy and Management 44 (1): 57–71. doi:10.1080/1360080X.2021.1934246.
Lee, M., D. Nasrawi, M. Hutchinson, and R. Lakeman. 2021. “Our Uni Teachers Were Already among
the World’s Most Stressed. COVID and Student Feedback Have Just Made Things Worse.The
Conversation, July 19. https://theconversation.com/our-uni-teachers-were-already-among-the-
worlds-most-stressed-covid-and-student-feedback-have-just-made-things-worse-162612
LeFebvre, L. E., H. J. Carmack, and J. R. Pederson. 2020. “I0t’s Only One Negative Comment”: Women
Instructors’ Perceptions of (Un) Helpful Support Messages following Hurtful Course Evaluations.
Communication Education 69 (1): 19–47. doi:10.1080/03634523.2019.1672879.
Mason, O., and N. Megoran. 2021. “Precarity and Dehumanisation in Higher Education.Learning and
Teaching 14 (1): 35–59. doi:10.3167/latiss.2021.140103.
Shepherd, S. 2018. “Managerialism: An Ideal Type.Studies in Higher Education 43 (9): 1668–1678. do
i:10.1080/03075079.2017.1281239.
Singh, C., W. Cross, I. Munro, and D. Jackson. 2020. “Occupational Stress Facing Nurse Academics—a
Mixed-Methods Systematic Review.Journal of Clinical Nursing 29 (5–6): 720–735. 10.1111/jocn.15150.
Spooren, P., B. Brockx, and D. Mortelmans. 2013. “On the Validity of Student Evaluation of Teaching:
The State of the Art.Review of Educational Research 83 (4): 598–642. doi:10.3102/0034654313496870.
Stein, S. J., A. Goodchild, A. Moskal, S. Terry, and J. McDonald. 2021. “Student Perceptions of Student
Evaluations: Enabling Student Voice and Meaningful Engagement.” Assessment & Evaluation in Higher
Education 46 (6): 837–851. doi:10.1080/02602938.2020.1824266.
Stroebe, W. 2020. “Student Evaluations of Teaching Encourages Poor Teaching and Contributes to
Grade Inflation: A Theoretical and Empirical Analysis.Basic and Applied Social Psychology 42 (4):
276–294. doi:10.1080/01973533.2020.1756817.
Uttl, B., and D. Smibert. 2017. “Student Evaluations of Teaching: Teaching Quantitative Courses Can
Be Hazardous to One’s Career. PeerJ 5: e3299. doi:10.7717/peerj.3299.
Wallace, S. L., A. K. Lewis, and M. D. Allen. 2019. “The State of the Literature on Student Evaluations
of Teaching and an Exploratory Analysis of Written Comments: Who Benefits Most?” College Teaching
67 (1): 1–14. doi:10.1080/87567555.2018.1483317.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Anonymous student evaluation of teaching (SET) is a universal practice in higher education. We conducted a mixed-methods approach to investigate the nature and impact of anonymous SET commentary in the Australian higher education sector. Respondents shared a range of detailed SET exemplars, which revealed the extent of hurtful, defamatory and abusive commentary made by students. This paper reports the self-perceived impact of these on the health and wellbeing of academics. The majority of respondents reported that anonymous narrative comments contributed to workplace stress. There were no significant differences for gender. Younger academics were more likely to report the process of SET as stressful. Four themes were identified from the narrative responses: stress, distress, disorder and coping. These themes highlight the mental distress and impacts on well-being from repeated exposure to uncivil commentary made in SET by students. This distress was exacerbated by the failure of many employing universities to take substantial action to remedy or limit exposure to uncivil behaviour. The current system of anonymous SET has little validity and instead may operate as a vehicle for unfettered incivility directed towards teaching staff. The mental health impacts are significant for some and may impact the recruitment, retention and renewal of academic teaching staff into the future.
Article
Full-text available
Within higher education student evaluations of teaching (SET) are used to inform evaluations of performance of courses and teachers. An anonymous online survey was constructed and implemented using Qualtrics. This study was situated within a more extensive study investigating the impact of narrative SET comments on teaching quality and the health and wellbeing of academic staff. This paper reports specifically on two open questions that were designed to elicit examples of non-constructive and offensive anonymous narrative feedback. Five themes were identified: allegations; insults; comments about appearance, attire and accent; projections and blame; and threats and punishment. These are represented in non-redacted form. Personally destructive, defamatory, abusive and hurtful comments were commonly reported. These kinds of comments may have adverse consequences for the well-being of teaching staff, could contribute to occupational stress and in some cases could be considered libellous. The high prevalence of offensive comments accessible to and shared by teachers may be a reflection of the anonymity afforded to respondents using internet surveys, resulting in de-individuation and enabling some respondents to give voice to ‘hate speech’ which has no place in evaluations of teaching.
Article
Full-text available
Australia’s higher education workforce has literally been decimated during the COVID-19 pandemic. Mass forced redundancies and non-renewal of casual contracts were highly stressful. And now some disciplines and academics who committed their lives to teaching feel publicly invalidated as unnecessary in the reconstruction of the sector to produce what the government deems to be “job-ready graduates”. Our recent review finds academics in Australia and New Zealand were suffering high levels of occupational stress well before COVID-19. Recent upheavals only added to existing problems. This is likely to jeopardise recruitment and retention of staff even in the very areas, such as health, teaching and medicine, where the government expects high future demand...
Article
Full-text available
The increased reliance of universities on a pool of highly skilled but poorly paid casualised academic labour for teaching and research has emerged as a defining feature of higher education provision under neoliberal New Public Management. Based on seventeen visual timeline interviews with academics in the North East of England, this article augments and extends existing studies of precarity through a framing of dehumanisation and humanisation. Specifically, we suggest that casualisation is dehumanising in four ways: it renders individuals invisible; it leaves them vulnerable to exploitation; it denies them academic freedom; and it hampers them in constructing a life narrative projecting into the future. We conclude that casualisation is not simply the product of a reprehensible political economy, but that it is an afront to the very meaning and dignity of being human.
Article
Full-text available
This paper analyses the current research regarding student evaluations of courses and teaching. The article argues that student evaluations are influenced by racist, sexist and homophobic prejudices, and are biased against discipline and subject area. This paper’s findings are relevant to policymakers and academics as student evaluations are undertaken in over 16,000 higher education institutions at the end of each teaching period. The article’s purpose is to demonstrate to the higher education sector that the data informing student surveys is flawed and prejudiced against those being assessed. Evaluations have been shown to be heavily influenced by student demographics, the teaching academic’s culture and identity, and other aspects not associated with course quality or teaching effectiveness. Evaluations also include increasingly abusive comments which are mostly directed towards women and those from marginalised groups, and subsequently make student surveys a growing cause of stress and anxiety for these academics. Yet, student evaluations are used as a measure of performance and play a role in hiring, firing and promotional decisions. Student evaluations are openly prejudiced against the sector’s most underrepresented academics and they contribute to further marginalising the same groups universities declare to protect, value and are aiming to increase in their workforces.
Article
Full-text available
Student evaluations of teaching are ubiquitous in the academe as a metric for assessing teaching and frequently used in critical personnel decisions. Yet, there is ample evidence documenting both measurement and equity bias in these assessments. Student Evaluations of Teaching (SETs) have low or no correlation with learning. Furthermore, scholars using different data and different methodologies routinely find that women faculty, faculty of color, and other marginalized groups are subject to a disadvantage in SETs. Extant research on bias on teaching evaluations tend to review only the aspect of the literature most pertinent to that study. In this paper, we review a novel dataset of over 100 articles on bias in student evaluations of teaching and provide a nuanced review of this broad but established literature. We find that women and other marginalized groups do face significant biases in standard evaluations of teaching – however, the effect of gender is conditional upon other factors. We conclude with recommendations for the judicious use of SETs and avenues for future research.
Article
Student evaluation of teaching (SET) surveys are the most widely used tool for collecting higher education student feedback to inform academic quality improvement, promotion and recruitment processes. Malicious and abusive student comments in SET surveys have the potential to harm the wellbeing and career prospects of academics. Despite much literature highlighting abusive feedback in SET surveys, little research attention has been given to methods for screening student comments to identify and remove those that may cause harm to academics. This project applied innovative machine learning techniques, along with a dictionary of keywords to screen more than 100,000 student comments made via a university SET during 2021 Higher Education Standards Framework (Threshold Standards). 2021. (Cth) (Austrl.) [Google Scholar]. The study concluded that these methods, when used in conjunction with a final stage of human checking, are an effective and practicable means of screening student comments. Higher education institutions have an obligation to balance the rights of students to provide feedback on their learning experience with a duty to protect academics from harm by pre-screening student comments before releasing SET results to academics.
Chapter
Introduction In the last two decades, insecure work in universities has grown exponentially in many countries, alongside the rapid marketisation of higher education. Reflecting the neoliberal ideal of a flexible workforce, research and teaching is now routinely carried out by precariously employed, hourly paid academics. In Australia, where we work, the bulk of teaching is now carried out by these ‘casual academics’ (Coates and Goedegebuure, 2010). Of course, casual teachers are needed to meet short-term needs – to fill in for sick staff, give one-off specialist lectures or provide insights into current practices, for instance – however, they should not be used to meet ongoing teaching needs. Structural dependence on casual academics poses a range of risks. It undermines the academic career path, and thereby the profession as a whole; threatens the quality of university education, posing a reputational problem for the sector and individual universities; and creates industrial injustice for casual academics who are locked into a permanently insecure marginal status. In this chapter, we focus on the possibilities for challenging casualisation through ‘statistical activism’, to force stakeholders to address these risks. Our interest in these possibilities arises from a problem we encountered in a study on the impacts of a new entry-level category of academics designed to reduce casualisation. The introduction of this employment category was a milestone in the ongoing campaign of the university staff union, the National Tertiary Education Union (NTEU), to address the industrial injustices experienced by its casual members, specifically the failure by universities to provide: mechanisms for conversion into continuing positions, paid leave, recognition of scholarship and research contributions, and rights to advancement on pay scales. From 2012 onwards, the creation of these positions was agreed during enterprise bargaining in many universities across Australia, reflecting acknowledgement by both the university managements (Hugo and Morriss, 2010) and the NTEU (2016) that the significant growth in precarious employment in the sector was neither sustainable nor desirable. For both managers and union activists to meaningfully assess the impact of these new positions on casualisation, reliable estimates of casualisation trends are needed. Yet the statistics on casualisation presented by universities and the Department of Education (DET) present a partial picture.
Article
Occupational stress has increased in higher education academic staff over several decades, and this has been particularly acute in Australia and New Zealand. This scoping review sought to understand the causes and impacts of occupational stress among Australian and New Zealand academics. Eight EBSCO databases were searched for key terms: academic and occupational stress and Australia and New Zealand. Twenty relevant papers were sourced, from which five common themes were extracted: (i) balancing an academic workload, (ii) casualisation of the workforce, (iii) the managerialism phenomenon, (iv) transition from field of practice to academia, and (v) academic and other staff. Further research in the Australian and New Zealand context is required to identify the nature of specific stressors and how these impact health and well-being.